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CONARG  Training  Workshop 
5-7  October  1971 


Systems  Engineering  in  Training 
Session  #1  (90  Min) 

"Systems  Engineering  Within  the  Armed  Forces-A  Status  Report" 


Army  Major  Tamer 

Marine  Corps  LTC  Godfrey 

Navy  Dr.  Havens 

Air  Force  Mr.  Neal 

Break 

element  will  present  a  brief  but  compre¬ 
hensive  status  report  concerning  the  utilization  of  systems 
engineering  as  a  process  for  the  design  of  curriculum.  Pre¬ 
sentation  will  identify  problem  areas  and  successes  in  the 
development  and  growth  of  the  systems  approach  to  course 
design. 


5  Oct 
U*00-li*25 
11*25-11 *1*5 
11*1*5-1505 
0505-1530 
^53Q-l55o 


6  Oct 

11*00-11*25 

11*25-11*1*5 

11*1*5-1505 

1505-1530 

1530-1550 


SCOPE:  Each  service 


"Overview  of  Course  Design  through  the  Systems  Approach  to 
Cirriculum  Development— a  Survey  of  Contemporary  Research" 


5  Oct 
1550-1610 


6  Oct 
1550-1610 


•N 


"Systems  Approach  to  Course 


Design  in  the  Civilian  Community" 
Dr.  Gagne 

1610-1700  1610-1700  *Panel  and  Open  Forum  on  Issues 

and  Problems  in  Systems  Engineering 
Major  Tamer 
Mr.  Vassos 

-*Panel  will  be  composed  of  representatives  of  the  armed  forces 
and  civilian  community 


XOPE:  An  internationally  known  research  specialist  will  review 
the  development  and  current  direction  of  the  systems  approach  to 
course  design  in  the  civilian  community.  This  presentation  will 
be  followed  by  a  panel  of  representatives  from  the  armed  forces 
and  civilian  communities  who  will  respond  to  questions  in  an 
open  forum.  Additionally,  comments/responses  to  the  systems 
engineering  field  survey  conducted  by  USATSCH  will  be  indent! fied 
and  introduced  for  workshop  discussion. 
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"Unique  Procedures  and  Techniques  in  Course  Design  through 
Systems  Engineering" 


6  Oct 
0830-0850 

0850-0910 

0910-0930 

0930-0950 

0950-1010 

1010-1030 

1030-1100 


7  Oct 
0830-0850 

0850-0910 

0910-0930 

0950-0950 

0950-1010 

1010-1030 

1030-1045 


Systems  Approaches  to  the  Design  of 
Curriculum  Mr.  Foster 

Administrative  Innovations 

Dr.  Tiemann 

Soft  Skill  Courses  Mr.  Harvey 

Computerized  Aspects  of  Systems 
Eng ineer ing  Mr .  Henry 

Course  Structure  Concept 

Dr.  Gray 

Open  Forum  Period  Major  Tamer 

Mr.  Vassos 


Break 


SCOPE:  Presentations  by  Army  Service  Schools  will  highlight 
unique  procedures  and  techniques  utilized  in  the  system  engi¬ 
neering  process.  An  open  forum  will  follow  to  gain  further 
information  and  insights  concerning  these  aspects  to  systems 
engineering.  The  presentations  of  Workshop  Session  03  are 
keyed  to  the  response  and  requests  resulting  from  the  systems 
engineering  field  survey  conducted  by  the  USATCH. 

"Information  Interchange  and  Future  Directions  in  Systems 
Engineering" 


6  Oct 

7  Oct 

1100-1130 

1045-1115 

Commercial  Approach  to  Systems 

Engineer ing 

Dr.  Hoyt 

1130-1145 

1115-1130 

Discussion 

Major  Tamer 
Mr.  Vassos 

1145-1155 

1130-1140 

Views  from  CONARC 

Mr.  McDowell 

1155-1215 

1140-1200 

Workshop  Observations 

Dr.  Gagne 

1215-1400 

1200-1330 

Lunch 

1330-1500 

Plenary  Session 

Workshop  Summary  and  Critique 

1500 

Adjournment 

SCOPE:  A  presentation  by  this  system  development  corporation 
identifying  a  unique  approach  to  the  systems  engineering 
function.  Future  directions  and  comments  will  be  explored  in 
views  from  00NARC.  Dr.  Gagne  will  provide  professional  ob¬ 
servations  concerning  attainment  of  workshop  objectives 
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Foster,  Mr  Ji— •  L. 

US  Air  Defense  School 
ATTN:  DOI 

Fort  Bliss,  Texas  79916 

Gagne,  Dr  Robert  M. 

Dept  of  Educational  Research 
College  of  Education 
Florida  State  University 
Tallahassee,  Florida  23306 

Garneau,  LTC  Luclen  R. 

US  Any  Transportation  School 

ATTN:  ATSTC-IC 

Fort  Eustls,  Virginia  23604 

Godfrey,  LTC  Edwin  J. 
Headquarters,  US  Marine  Corpa 
ATTN:  Code  A03C 
Washington,  D.  C.  20380 

Gray,  Dr  Charles  0. 
Educational  Advisor 
US  Any  Engineer  School 
Fort  Relvolr,  Virginia  22060 

Harvey,  Mr  Charles 
US  Any  Infantry  School 
ATTN:  DOI 

Fort  Banning,  Gaorgla  31905 

Havana,  Dr  Charles 
Chief,  Naval  Air  Training 
Naval  Air  Station 
Pensacola,  Florida  32508 

Henry,  Mr  Ja— a  E. 

Educational  Advisor 
US  Any  Quartermaster  School 
Fort  Lae,  Virginia  23801 


Hoyt,  Dr  Willi— 

Systa—  Develop— nt  Corporation 

2500  Colorado  Avenue 

Santa  Monica,  California  90406 

McDowell,  Mr  Walter  E. 

Hq  CON ARC 

ATTN :  ATIT-STM 

Fort  Monroe,  Virginia  23351 

Myers,  Mr  John  R. 

Ed  Spec,  Curr  Div,  DOI 
US AS ESS 

Fort  Gordon,  Georgia  30905 

Neal,  Mr  Willi—  S. 

Co— —ding  Officer 
3750th  Training  Squadron 
ATTN:  OPNS  Div  TTO/20 
Sheppard  AFB,  Tex—  76311 

Ta—  r,  MAJ  Robert  S. 

US  Any  Transportation  School 

ATTN :  ATSTC-ICS 

Fort  Eustls,  Virginia  23604 

Tie— nn.  Dr  P.  W. 

US  Any  Medical  Dept  Vet  School 
1749  W— t  Pershing  Road 
ATTN:  EA 

Chicago,  Illinois  60609 

Vassos,  Mr  Aga— on 
Ed  Spec 

US  Any  Transportation  School 

ATTN:  ATSTC-ICS 

Fort  Bustle,  Virginia  23604 
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BIOGRAPHICAL  SKETCH  OF  SYSTEMS  ENGINEERING 


CONSULTANTS 


Dr.  Philip  V.  Tiemann  -  A  education  advisor  to  the  U.  S.  Army 
Veterinary  School  and  a  faculty  member  of  the  University  of 
Chicago. 

Dr.  Robert  M.  Gagne  -  A  professor  in  the  Department  of  Educational 
Research  and  Testing  at  Florida  State  University.  He  has  been 
actively  engaged  in  research  on  human  learning  for  many  years  and 
is  regarded  as  a  leading  authority  in  the  field.  His  most  recent 
boohs  are  The  Conditions  of  Learning  and  Learning  and  Individual 
Differences. 
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SYSTEMS  ENGINEERING  WORKSHOP  ORIENTATION 
(An  Introduction) 


LTC  Lucien  R.  Gameau 

U.  S,  Aimy  Transportation  School 

Port  Eustis,  Virginia  2360b 


z. 


systems  engineering  workshop  orientation 

(An  Introduction) 

LTC  Lucien  R.  Garneau,USA 

The  1)/  S.  Army  Transportation  School  is  proud  to  be  a  part 
of  this  conference  and  to  have  been  selected  to  conduct  the 
Systems  Engineering  Workshop.  Perhaps  I  can  set  the  stage  for 
the  Systems  Engineering  Workshop  by  relating  a  fable  which  is 
making  the  circuit  in  the  Systems  Engineering  Field. 

A  Fable  by  Roger  A.  Kaufman 

Once  upon  a  time  there  were  two  pigs  (a  third  one  had  gone 
into  marketing  and  disappeared)  who  were  faced  with  the  problem 
of  protecting  themselves  from  a  wolf. 

One  pig  was  an  old-timer  in  this  wolf-fending  business,  and 
he  saw  the  problem  right  away  —  just  build  a  house  strong  enough 
to  resist  the  huffing  and  puffing  he  has  experienced  before.  So, 
the  first  pig  built  his  wolf-resistant  house  right  away  out  of 
genuine,  reliable  lath  and  plaster. 

The  second  pig  was  green  at  this  wolf  business,  but  he  was 
thoughtful.  He  decided  that  he  would  analyze  the  wolf  problem 
a  bit.  He  sat  down  and  drew  a  matrix  (which,  of  course,  is  pig 
latin  for  a  big  blank  sheet  of  paper)  and  listed  the  problem, 
analyzed  the  problem  into  components  and  possibilities  of  wolf 
strategies,  listed  the  design  objectives  of  his  wolf-proof 
house,  determined  the  functions  that  his  fortress  would  perform, 
designed  and  built  his  house,  and  waited  to  see  how  will  it 
worked.  (He  had  to  be  an  empircist,  for  he  had  never  been  huffed 
and  puffed  at  before.) 

All  this  time,  the  old-timer  pig  was  laughing  at  the  planner 
pig  and  vehemently  declined  to  enter  into  this  kind  of  folly.  He 
had  built  wolfproof  houses  before,  and  he  has  lived  and  prospered, 
hadn’t  t e?  He  said  to  the  planner  pig,  "If  you  know  what  you  are 
doing,  you  don’t  have  to  go  through  all  that  jazz."  And  with  this, 
he  went  fishing,  or  rooting,  or  whatever  it  is  that  pigs  do  in  their 
idle  hours. 

The  second  pig  worked  his  system  anyway,  and  designed  for  pre¬ 
dicted  contingencies. 

One  day  the  mean  old  wolf  passed  by  the  two  houses  (they  both 
looked  the  same  —»  after  all,  a  house  is  just  a  house.)  He  thought 
that  a  pig  dinner  was  just  what  he  wanted.  He  walked  up  to  the 
first  pig’s  house  and  uttered  a  warning  to  the  old  timer,  which 
was  roundly  rejected,  as  usual.  With  this,  the  wolf,  instead  of 
huffing  and  puffing,  pulled  out  a  sledge  hammer,  knocked  the  door 
down,  and  ate  the  old  timer  for  dinner. 

Still  not  satiated,  the  wolf  walked  to  the  planner  pig’s  house 
and  repeated  his  act,  Suddently,  a  trap  door  in  front  of  the  house 
opened  and  the  wolf  dropped  neatly  into  a  deep,  dark  pit,  never  to 
be  heard  from  again. 

Morals:  1.  They  are  not  making  wolves  like  they  used  to. 

2.  It’s  hard  to  teach  old  pigs  new  tricks. 

3.  If  you  want  to  keep  the  wolf  away  from  your 
door,  you’d  better  system  engineer. 
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The  pig  fable  gives  us  the  rationale  for  Systems  Engineering. 

How  I  would  ltxe  to  give  you  the  rationale  that  the  Transportation 
School  used  in  planning  the  Systems  Engineering  Workshop. 

First  of  all,  we  took  the  primary  purpose  of  the  conference 
which  was  to  "foster  an  extensive  interchange  of  ideas  and  inf or* 
mation  relating  to  innovations  in  training"  and  built  this  into 
the  program  structure.  This  was  accomplished  by  inviting  part* 
icipants  from  the  Armed  Forces,  industry  and  the  civilian  comm¬ 
unity.  Also,  within  the  Army,  we  brought  together  the  talents 
of  several  service  schools  to  provide  the  breath  of  interchange 
we  were  seeking.  ” 

Since  we  are  highly  pleased  and  proud  of  the  Systems  Engi¬ 
neering  team,  the  Transportation  School  has  assembled,  I  would 
like  to  share  with  you  our  list  of  distinguished  participants. 

From  the  Service  Schools  we  haves 
Mr.  Foster  «  Air  Defense  School 
Dr.  Tiemann  -  Veterinary  School 
Mr.  Harvey  -  Infantry  School 
J|lr.  Henry  -  Quartermaster  School 
Dr.  Cray  -  Engineer  School 
Major  Tamer  -  Transportation  School 
Mr.  Vassos  «  Transportation  School 
From  CONARC  we  have  incited: 

Mr.  McDowell 

From  industry  we  have 

Dr.  Hoyt  from  the  System  Development  Corp. 

From  the  other  Armed  Services  we  have: 

LTC  Godfrey  -  U«  S.  Marine  Corps 
Dr.  Havens  -  Navy 
Mr.  Neal  -  Air  Force 

Additionally,  we  are  pleased  to  have  Dr.  Gagne,  an  internationally 
known  figure  in  education  research  from  Florida  State  University, 
represent  the  civilian  community. 

Also,  many  of  these  distinguished  people  have  brought  some  of 
their  colleagues  to  assist  them  in  their  presentations  and  act  as 
resource  people  to  the  participants  of  the  Systems  Engineering 
Workshop. 

Secondly,  we  wanted  the  Systems  Engineering  Workshop  to  re¬ 
flect  the  latest  thinking,  techniques,  procedures  and  accomplish¬ 
ments  in  the  Systems  Engineering  field. 

This  was  accomplished  by  an  extensive  field  survey  consisting 
of  a  structured  questionaire  sent  to  CONARC  activities  to  gather 
the  data  necessary  to  meet  this  objective. 

After  analyzing  the  results  of  the  field  survey  we  were  able 
to  identify  and  select  areas  of  interest  and  concern  prinary  to 
those  elements  working  in  the  field  of  systems  engineering. 
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As  a  note  of  interest,  it  was  amazing  that  the  results  of 
the  survey  identified  a  high  degree  of  commonality  in  restricted 
areas  of  interests. 

So,  we  built  into  the  Systems  Engineering  Workshop  as  many 
of  these  areas  as  time  would  allow. 

Additionally,  we  included  this  infoimation  in  the  handout 
material  which  will  be  found  in  the  portfolios  you  will  receive 
as  a  participant  at  the  Systems  Engineering  Workshop.  This  was 
done  so  that  you  may  be  aware  of  these  areas  and  seek  resolution 
of  your  interests  either  during  this  conference  or  in  your 
follow-up  activities  after  the  conference. 

This  leads  us  to  the  third  element  of  our  rationale  -  Inter¬ 
action  and  future  activity. 

Total  resolution  of  problem  areas  and  development  of  area 
interests  cannot  be  hoped  for  during  this  conference.  Realizing 
this,  we  suggest  two  things  which  might  help  you  in  this  area. 

First,  use  every  spare  minute  to  interest  and  seek  resolution 
whenever  you  can.  Don't  be  cliquish  and  stick  with  your  colleagues 
that  came  to  the  conference  with  you,  but  have  meals  and  rap 
sessions  with  those  individuals  who  can  help  you  reach  the  ob¬ 
jectives  for  which  you  are  striving. 

Secondly,  realizing  again  that  our  time  is  limited,  we  will 
compile  a  Directory  of  Participants  in  the  Systems  Engineering 
field  whom  you  may  contact  at  your  leisure  as  residual  activity 
of  our  workshop  to  cultivate  a  relationship  which  will  be  of 
mutual  benefit. 

The  last  element  in  our  program  rationals  was  the  indent- 
ification  of  resources.  Our  most  important  resource  is  people 
and  we  are  proud  to  bring  you  the  best  that  we  could  find  to 
make  this  the  most  professional  workshop  of  it's  kind. 

Also,  we  recognize  that  the  more  we  know  about  our  work, 
the  more  we  will  be  able  to  do.  So,  we  have  included  a  tdblio- 
graph  of  selected  publications  which  will  stimulate  your  think¬ 
ing  and  provide  you  with  information  necessary  to  perform  your 
duties  more  effectively. 

Therefore,  you  are  encouraged  to  share  your  findings  along 
these  lines  in  order  that  we  can  add  more  to  our  warehouse  of 
resources. 

This  then  gives  you  some  insight  into  what  you  can  expect 
from  the  systems  engineering  workshop. 

The  Transportation  School  has  assembled  the  best  talent 
available  in  the  field  of  Systems  Engineering.  We  have  even 
applied  the  principles  of  Systems  Engineering  in  the  planning 
of  the  Wsrkshop.  So  you  know  it  has  to  be  good. 

Now,  there  is  only  one  ingredient  left  to  be  added  to  the 
Systems  Engineering  Workshop  to  make  it  one  of  the  most  mean¬ 
ingful  experiences  for  you  and  that  is  your  participation. 

Henry  Ford  expressed  this  extremely  well  when  he  said: 

"Coming  together  is  a  beginning;  keeping  together  is  progress; 
working  together  is  success."  Now,  we  look  forward  to  your 
participation. 


SYSTEMS  ENGINEERING  WITHIN  THE  U.  S.  ARMY 
A  Status  Report 


Major  Robert  S.  Tamer 

U.  S.  Army  Transportation  School 

Tbrt  Eustis,  Virginia  23601* 


Good  Afternoon,  I*m  Major  Tamer  of  the  Transportation  School 
at  Fort  Fustis.  I  would  like  to  direct  your  attention  to  the  TV 
monitors  for  a  short  introductory  tape.  As  you  see  this  tape, 
consider  in  your  own  mind  a  possible  title  of  "Why  Systems  Eng- 
ineering,"  or  was  a  new  approach  to  curriculum  development  needed. 
Although  amusing,  the  tape  identified  two  of  the  main  reasons 
demanding  a  sophisticated  approach  to  curriculum  development  with¬ 
in  the  Army.  Graphically  illustrated  were  the  technological 
advances  in  military  hardware  and  the  need  for  advances  in  educa¬ 
tional  technology  within  Army  training  programs.  The  result  of 
this  recognition  was  the  creation  of  the  systems  engineering 
function  within  the  continental  Army  Command.  But  before  going 
further  what  is  systems  engineering  in  the  Army?  In  an  attempt 
to  bring  us  all  to  common  ground,  I  feel  a  brief  explanation  may 
be  in  order  due  to  the  many  concepts  and  views  existing  covering 
this  subject.  Systems  engineering  as  a  function  includes  all  of 
the  elements  we  formerly  identified  in  curriculum  development. 

Now  these  elements  are  formalized  in  a  systematic  and  fully  docu¬ 
mented  approach  to  curriculum  development,  or  simply  -  A  total 
approach  to  subject.  But  what  does  this  process  involve? 

Systems  engineering  involves  a  through  investigation  into 
every  factor  or  potential  factor  influencing  an  individual  sold¬ 
ier's  ability  to  perform  his  job  at  a  required  level  in  the  field; 
it  involves  interaction  and  coordination  with  all  governmental  and 
nongovernmental  activities  charged  with  the  actions  you  see  here 
such  as  doctrine,  plans  and  equipment.  This  activity  culminates 
in  follow-up  research  in  an  attempt  to  fully  analyze  the  job  tasks 
required  of  the  individual  soldier,  and  provides  the  basis  for 
what  must  be  taught  and  more  important,  what  must  be  learned. 

Curriculum  developed  utilizing  this  dynamic  process  results 
in  greater  eff ic iency  in  the  utilization  of  resources,  (both 
material  and  personnel;)  greater  effectiveness  in  constructing 
courses  of  instruction  solidly  based  around  those  essential  elements 
of  the  job  identified  in  the  field;  in  the  structuring  of  courses 
best  for  learning  and  insures  that  the  most  effective  instructional 
techniques,  enviroment,  methods  and  media  are  utilized  and  fully 
integrated.  All  this  then,  results  in  the  conservation  of  all 
resources,  training  time  and  directs  our  instructional  emphasis. 

Some  of  the  spinoffs  of  the  systems  engineering  process  include 
a  purification  of  the  job  most  structure  and,  curriculum  objectivity 
to  insure  that  biases,  whether  personally  or  professionally  moti- 
vated,  do  not  appear  in  the  instructional  process.  Further  this 
process  provides  a  dynamic  vehicle  with  which  to  individualize  the 
instructional  process  at  the  grass  roots  level  —  A  key  objective 
within  Gen  Haine's  speech. 


VI 1-9 


All  this  then  is  directed  at  the  single  purpose  of  systems 
engineering;  —  To  develop  the  best  training  program  possible 
to  prepare  soldiers,  our  graduates,  to  perform  at  the  highest 
level  of  productivity  possible.  So  much  for  defining  systems 
engineering  within  the  Army;  certainly  it  was  a  cursory  overview 
of  a  highly  technical,  and  detailed  process. 

But  let*s  now  look  at  some  of  the  factors  influencing  systems 
engineering  development  with  the  Army.  Systems  engineering  within 
CONARC  training  activities  has  its  birth  in  February  of  1968  with 
the  publication  of  CONARC  regulation  350-100-1.  The  graphically 
portray  this  development,  I  will  draw  on  analogy  between  the  growth 
of  systems  engineering  and  the  growth  of  a  child.  Since  this  is 
common  to  all  of  us  the  transition  should  be  easier. 

Let*s  call  our  baby  Systems  Engineering.  He  was  born  during 
a  turbulent  period  within  the  Army;  one  in  which  the  Amy  was  in¬ 
volved  in  a  process  of  mobilization  and  mission  orientation  hereto¬ 
fore  unheralded  in  our  military  history.  And  as  all  babies  do  — 
systems  engineering  initially  reflected  the  sum  total  of  its  environ¬ 
ment.  This  environment  was  dominated  by  our  Vietnam  Mission 
Commitment  and  Selective  Service  Mobilization  Program  which  developed 
around  C0NARC*S  ability  to  train  and  provide  soldiers  with  the 
requisite  skills  necessary  to  perform  immidiately  at  the  entry  level 
of  their  MOS.  This  mammoth  responsibility  was  directed  at  pro¬ 
viding  graduates  which  required  little  or  no  on  the  job  training. 

We  could  no  longer  depend  upon  field  units  to  conduct  on  the  job 
training  to  round  out  a  soldier*s  school  training  and  qualifications. 
Every  element  within  the  Army  was  completely  and  totally  devoted 
to  their  mission  commitments  whether  in  Vietnam  or  in  Europe  where, 
by  the  way,  our  personnel  draw  down  made  it  necessary  for  one  soldier 
to  perform  as  two.  CONARC  was  charged  with  the  challenge  of  producing 
soldiers  capable  of  entry  level  productivity  upon  graduation.  "Hands 
on  training"  was  emphasized  and  this  particular  phrase  became  the 
focal  point  for  commanders  and  military  educators  alike.  Additionally, 
new  advances  and  availability  of  educational  technology  made  a  new 
and  totally  integrated  approach  to  curriculum  development  both  ne¬ 
cessary  and  mandatory. 

Now  as  with  everything  new,  we  had  a  number  of  problems  in 
developing  our  new  baby  —  Systems  Engineering.  First  of  all, 
the  terminology  utilized  in  describing  the  process  and  its  inter¬ 
pretation  fell  on  untrained  senses.  How  do  we  use  criterion, 
standard,  matrix,  etc.  and  more  important  what  do  they  mean?  The 
development  of  organizational  concepts  to  conduct  the  function 
whether  decentralized  or  centralized,  was  purely  trial  and  error. 

Some  00NARC  elements  established  separate  systems  engineering 
branches  and  dividions,  while  others  intergrated  the  function  within 
the  existing  organization.  Manpower  needs  as  well  as  manpower 
authorizations  and  justifications  were  unknown  and  vague.  Command 
emphasis  was  a  necessity  but  too  much  could  have  resulted  in  a 
total  production-oriented  systems  engineering  program  directed  at 
developing  facsimilies  or  regurgitated  courses  of  instruction. 
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If  and  when  this  were  to  happened,  systems  engineering  would 
be  reduced  to  nothing  more  than  a  new  label  or  title.  Since  this 
new  baby  contained  the  majority  of  curriculum  development  responsi¬ 
bilities  previously  found  within  other  established  organizational 
elements,  as  well  as  those  solely  peculiar  to  systems  engineering, 
duplication  of  functions  was  a  fact.  Defining  and  isolating  staff 
responsibility  for  the  all-encompassing  seven  systems  engineering 
steps  was  necessary  if  manpower  problems,  organizational  structure, 
and  the  development  of  standard  operating  procedures  were  to  be 
realized.  Additionally,  it  was  imperative  that  formalized  training 
of  project  engineers  and  the  staff  and  faculty  be  initiated  if 
systems  engineering  were  to  achieve  its  full  potential.  These 
problems  were  magnified  by  the  initial  resistance  and  lack  of  res¬ 
ponsiveness  of  the  supporting  informational  sources,  so  necessary 
to  complete  an  adequate  job  analysis  -  and  if  all  these  problems 
were  not  enough  to  stump  the  finest  systems  engineering  pediatrician, 
our  baby  was  being  staffed  and  restaffed  just  about  every  12  months 
due  to  the  personal  turbulence  associated  with  the  rapid  turnover 
of  project  engineers  to  meet  mobilization  requirements.  This  one 
problem  may  be  the  root  cause  for  our  systems  engineering  baby 
failing  to  meet  the  initial  five  year  goals  identified  in  CONARC 
reg  350-100-1  when  considering  the  long  duration  required  to  systems 
engineer  a  course  of  instruction,  compounded  by  the  long  training 
and  experience  required  to  make  a  project  engineer  productive,  the 
rapid  personel  turnover  problem  resulted  in  excessive  expenditures 
of  all  resources  and  completely  colored  our  badies  progress.  A 
good  example  of  this  problem  could  be  deducted  from  the  three 
years  required  to  redesign  the  turbine  engine  repairman  course  of 
instruction  and  the  10  project  engineers  assigned  the  project 
during  this  period.  This  course  could  have  been  redesigned  in  one 
year  by  one  project  engineer.  But,  as  Robert  Ripley  would  say 
"Believe  it  or  not,"  our  baby,  systems  engineering,  grew  to  child¬ 
hood  -  and  after  two  years  of  activity.  Papa  CONARC  called  for 
and  reviewed  his  first  report  card.  Papa  CONARC  dosen't  appear 
to  be  pleased  -  Let's  have  a  look  at  our  boys  process. 

Well,  I  think  this  report  card  graphically  portrays  the 
crystallization  period  of  systems  engineering  -  as  well  as  some 
of  the  ills  still  plaguing  systems  engineering's  ability  after 
two  years  to  make  the  honor  roll.  Although  we  appear  to  have 
a  failure  in  one  area,  this  conference  should  improve  our  grade 
in  this  area.  However,  an  overall  passing  grade  in  identified, 
including  some  of  the  areas  responsible  for  this  achievement. 

Also  during  this  period,  there  was  a  plus  factor  in  the  tremendous 
awareness  and  confidence  growing  in  the  products  ot  systems 
engineering.  I  think  the  teachers  comment  bears  noting.  As 
with  all  children,  the  inevitable  occurred,  in  late  1970,  as 
systems  engineering  arrived  at  limited  productivity  and  adoles¬ 
cence.  It  was  during  this  period  that  Papa  CONARC  found  the 
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guidance  directing  the  systems  engineering  function  inadequate. 

We  how  began  to  face  the  facts  of  life  identifying  a  need  for 
new  guidance  and  greater  interaction.  As  a  result,  the  Director 
of  Instruction/Education  Advisor  Conference  of  March  of  this 
year.  Papa  Conarc  recommended  to  the  Chief  of  Staff  of  the  Army 
the  workshop  we  are  now  involved  in  come  about.  All  Army 
agencies  charged  with  implementing  the  systems  engineering  funct¬ 
ion  have  achieved  a  degree  of  success  and  reached  various  levels 
of  maturity.  Additionally,  each  of  these  systems  engineering 
agencies  individually  resolved  problems  in  streamlining  procedures 
through  standardization;  developed  new  techniques  including  the 
increasing  use  of  automatic  data  processing;  broadened  perspectives 
in  total  Army  interaction;  and  shared  the  products  of  systems 
engineering  both  vertically  and  horizontally  within  the  Army 
organizational  structure.  This  increased  activity  has  led  to 
functional  refinement  of  a  quality  product  and  the  recognition 
so  necessary  to  solve  our  manpower  and  organizational  difficulties. 

Now  the  stated  objectives  of  this  workshop  are  directed  at  sharing 
these  products  of  our  total  maturation.  Therefore,  it  is  imperative 
that  our  workshop  interaction  be  extensive  and  meaningful  and  that 
we  all  grow  to  higher  levels  of  productivity  as  a  result.  The  folders 
you  have  received  as  a  participant  contain  the  catalistic  elements 
of  this  sharing.  We  at  the  Transportation  School,  have  found  that  the 
use  of  a  comprehensive  checklist  as  part  of  our  systems  engineering 
standard  operating  procedure  insures  product  uniformity.  We  also  use 
an  Automatic  Data  Processing  program  which  resulted  in  an  estimated 
60  man  day  time  savings.  These  are  only  two  examples  of  the  elements 
of  your  packets  which  you  may  find  benedicial  and  useful.  Our  process 
of  maturation  has  firmly  established  systems  engineering  in  the  CONARC 
family  but  how  do  we  arrive  at  adulthood? 

That’s  a  question  we  may  find  in  a  crystal  ball.  But  I  prefer  to 
think  a  more  approach  will  eminate  from  this  assembly.  But  where  is 
systems  engineering  now?  In  a  pure  numerical  score  card  tally,  we 
appear  to  have  come  a  long  way  in  meeting  our  systems  engineering  ob¬ 
jectives  by  completing  240  courses  of  instruction  out  of  a  total  of 
720.  But  what  does  this  tally  really  indicate  when  considering  the 
tremendous  and  rapid  changes  coming  about  in  the  orientation  of  the 
Army  today  which  will  necessitate  a  change  in  our  training  philosophy. 
Increasing  opportunities  for  unit  preparedness  training  or  on  the  job 
training,  the  emerging  volar  concepts  and  increasing  demands  for  re¬ 
sources  conservation  demand  we  begin  to  re-examine  what  we  have  done 
as  well  as  what  is  yet  to  be  done  to  insure  our  products  are  economical, 
purposeful  and  reflects  this  new  thinking.  Additionally,  we  should 
begin  to  consider  the  joint  development  of  school  courses  of  instruction 
with  formalized  systems  engineered  on  the  job  training  packages  to 
assist  and  enhance  unit  preparedness  training  programs.  In  this 
way,  we  can  insure  that  combat  readiness  will  not  be  affected 
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by  reductions  in  school  training  time  or  the  inability  of  individual 
units  to  develop  or  perform  on  the  job  training.  Systems  engineering 
within  the  Army  is  now  in  the  process  of  growing  to  full  capability 
although  many  of  us  may  feel  at  times  that  systems  engineering, 
because  of  changing  influences,  is  in  3  state  of  Limbo.  I  assure 
you  this  is  not  the  case. 

Systems  engineering  is  the  dynamic  tool  by  which  our  changing 
military  commitments,  philosophy  and  volunteer  Army  concepts  can 
be  realized.  We  have  a  tremendous  challenge  before  us  and  I'm 
certain  we  can  meet  it. 
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SYSTEMS  ENGINEERING  IN  THE  U.  S.  MARINE  CORPS 


LTC  Edwin  J.  Godfrey 
U,  S.  Marine  Corps 
Washington,  D.C.  20360 


THE  MARINE  CORPS  AND  THE  SYSTBC  APPROACH  TO  TRAINING 

In  a  summary  of  a  systems  engineering  workshop  survey  conducted 
recently,  27  different  topic  areas  of  interest  were  noted.  Although 
it  was  an  Army  survey,  each  topic  represented  an  area  the  Marine 
Corps  has  struggled  with,  too.  Rather  than  address  any  specific 
topic  or  problem  area.  A  brief  overview  of  the  Marine  Corps*  effort 
with  the  "Systems  Approach"  is  believed  in  order. 

This  overview  with  address  three  basic  areas  where  the  Marine 
Corps  has  applied  or  is  applying  the  "Systems  Approach:"  Marine 
Corps  formal  schools;  unit  level  training;  and  finally  our  recently 
developed  computer-supported  training  management  system  called  Gentras. 
DEFINITION  OF  TBtMS 

Whether  we  address  the  "Systems  Engineering  of"  or  the  "Systems 
Approach  to"  training,  we  are  speaking  of  essentially  the  same  pro¬ 
cess.  There  will  be  some  difference  in  terms,  and  some  difference 
in  flow  charts;  but  what  we  are  really  talking  about  is,  instructional 
design  starting  with  the  identification  of  performance  requirements, 
and  working  back  tnrough  the  selection  of  learning  objectives,  measure¬ 
ment  criteria  and  instructional  strategies.  To  a  valid  and  meaningful 
program  of  instruction  which  we  continue  to  validate  through  evalu¬ 
ation  and  feedback.  That,  of  course,  is  an  overly  simple  definition, 
but  it  does  summarize  a  generally  uniform  view  of  the  "Systems  Approach" 
to  training.  An  even  simpler  view  is  quoted  below  from  a  Marine 
Corps  unit  level  training  management  directive; 

"1.  Training  is  not  an  end  in  itself,  it  is  a  means  of  achieving 
desired  performance  on  the  part  of  units  and  individuals. 

To  this  end,  the  "Systems  Approach"  to  training  is  simply 
an  orderly  and  logical  approach.  A  systematic  approach  to 
the  design  of  instruction  and  the  development  of  training 
programs. 

2.  In  a  unit  level  training  program,  training  should  not  be 
conducted  merely  for  the  purpose  of  accomplishing  a  given 
amount  of  training.  It  should  be  conducted  for  the  purpose 
of  raising  unit  or  individual  performance  from  current 
levels  of  performance  capability  to  desired/required  levels." 

In  the  case  of  our  formal  school  training  what  we  are  really  talk¬ 
ing  about  is  a  workable  means  to  bridge  the  gap  between  the  needs  of 
the  field,  and  what  is  taught  in  our  schools. 

FORMAL  SCHOOLS 

In  January  1969,  after  learning  many  lessons  from  the  Army  at  Ft. 
Devens,  the  Air  Force  at  Lackland,  and  a  number  of  our  better  known 
civilian  education  specialists,  the  Marine  Corps  published  an  order 
entitled  "Design  of  Courses  of  Instruction."  Although  at  many  formal 
schools  either  intuitively  or  through  local  policy.  A  form  of  systems 
approach  to  course  design  was  already  in  effect.  This  Marine  Corps 
orders  -  1510.23  -  presented  the  first  time  that  the  "Systems 
Approach"  was  directed  as  Marine  Corps  policy.  It  should  be  added 
that  although  this  order  applied  principally  to  formal  schools  con¬ 
ducting  enlisted  ground  training  after  boot  camp.  It  nevertheless 
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laid  the  groundwork  for  subsequent  expansion  into  unit  level  train* 
ing,  and  ultimately  into  all  areas  of  Marine  Corps  training  and  ed¬ 
ucation.  The  order  itself  addressed  such  topics  as:  The  new  concept 
and  its  terminology;  the  collection  of  job  data;  the  selection  of 
learning  objectives;  the  preparation  of  performance  objectives;  the 
perparation  of  criterion  measurements;  the  selection  of  course  con¬ 
tent;  the  selection  of  instructional  strategies;  the  evaluation  of 
course  content;  testing  and  analysis;  graduate  followup;  and  the 
preparation  of  programs  of  instruction. 

What  the  new  order  did,  in  effect,  was  require  that  all  enlisted 
(ground)  formal  schools,  over  a  several-year  period,  validate  their 
programs  of  instruction,  starting  with  what  would  amount  to  a  "Table 
Top"  job  analysis  process.  Then  working  back  through  the  systematic 
process  of  learning/performance  objective  selection,  instructional 
strategy  selection,  and  course  design/validation.  The  process  des¬ 
cribed  in  the  order  represented.  Understandably,  a  considerable 
undertaking  for  all  the  formal  schools  involved.  For  most,  it  was 
a  test  undertaken  with  great  reluctance,  since  it  was  an  in-house 
effort  which  has  to  be  accomplished  by  already  overtaxed  instructional 
staffs.  Results  have  varied  from  reasonably  satisfactory,  to  out¬ 
standing;  but  in  virtually  all  cases,  the  result  has  proven  well 
worth  the  effort.  The  quality  and  appropriateness  of  instruction  has 
improved,  and  in  many  cases  course  lengths  have  been  reduced. 

The  recent  course  validation/redesign  effort,  however,  has  in 
fact  been  a  means  which  is  serving  to  bridge  the  gap  for  our  formal 
schools,  from  the  traditional  approach  to  instructional  design.  To 
an  approach  which  can  be  supported  by  an  advanced  computer  supported 
training  model. 

GEOTRAS 

Let  me  explain  the  last  comment  in  more  detail.  Under  the  current 
program,  job  or  task  analysis  is  determined  by  "Table  Top"  analysis 
at  the  school  itself.  Although  it  may  seek  support  from  field  commands. 
Since  early  1970,  the  Marine  Corps  has  been  undertaking  by  separate 
action.  A  detailed  comprehensive  "Task  Analysis  Program?'  which  over 
a  several-year  period  is  conducting  an  occupational  field  by  occupational 
field  analysis  to  identify  those  skills  and  that  knowledge  actually 
required  on  the  job.  The  impact  of  this  effort  reaches  many  areas, 
only  one  of  which  is  training.  From  the  training  standpoint,  however, 
the  impact  is  significant  job  performance  requirements  will  be  clearly 
identified,  and  validated,  MOS  by  MOS.  This  in  turn  provides  the 
trainers  with  a  far  more  valid  statement  of  training  requirements  than 
our  school-level  "Table  Top"  analysis  could  possibly  give  us. 

In  order  to  effectively  utilize  the  task  analysis  output  i.e.. 

The  validated  job  data  •  we  have  recently  designed,  in  cooperation 
with  IBM,  a  computer-based  training  management  system  called  the 
General  Training  System,  or  Gentras.  What  Gentras  will  do,  is  close 
the  loop  between  those  who  manage  overall  training.  Those  who 
conduct  it,  and  those  who  use  the  end  product,  the  "Field".  Through 
maintenance  of  computer-based  job  data  files,  course  data  files, 
and  a  field  evaluation  feedback  routine  -  each  using  compatibility 
performance/learning  objective  terminology,  Gentras  is  designed  to 
assure  compatibility  of  job  performance  and  the  instruction  acutally 
presented  in  our  formal  schools. 
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The  task  analysis  program  makes  available  to  Headquarter  Marine  Corp. 
detailed  job  descriptions  and  occupational  field  structure.  Course 
information  is  provided  by  the  schools.  The  training  management 
system  provides  automated  support  tor  storing  raw  MOSt  course  and 
field  evaluation  data;  and  queries  against  this  data  base  provide 
the  capability  for  selecting  and  correlatingthis  information  to  produce 
meaningful  output.  Using  this  information.  Headquarter  Marine  Corp 
can  determine  apparent  deficiencies  in  training  and  will.  In  turn, 
recommend  changes  to  the  schools.  The  schools,  acting  upon  these 
recommendations  will  be  able  to  pleace  better  trained  personnel  in 
the  field.  Through  field  evaluation  of  recent  graduates,  and  by  re- 
surveying  field  units.  Headquarters  Marine  Corp  will  be  able  to 
further  evaluate  jobs,  job  structure  and  training  effectiveness, 
whereupon  the  cycle  may  be  repeated.  Field  evaluations  will  normally 
be  conducted  on  a  sample  basis;  however,  they  are  always  conducted 
for  new  courses  and  courses  known  to  be  deficient. 

Gentras  has  just  begun  implementation.  Using  as  a  pilot  a 
recently  restructured  occupational  field.  We  anticipate  incorporation 
of  all  ground  occupational  fields  and  associated  courses  by  the  fiscal 
year  73/74  time  frame. 

UNIT  LEVEL  TRAINING 

Our  eifort' to  bring  Marine  Corp  training  under  the  umbrella  of 
the  "Systems  Approach"  has  not  been  limited  to  formal  school  train¬ 
ing.  Earlier  this  year,  we  published  a  unit  level  training  manage¬ 
ment  directive  -  MCO  P1510.26  -  Which  served  to  translate  the  "Systems 
Approach"  and  our  more  complex  design  of  course  of  instruction  order 
into  guidance  which  was  feasible  and  meaningful  to  the  unit  commander 
and  his  staff  officers  involved  in  determining  training  requirements, 
and  planning/supervising  training  programs.  It  addressed  such  areas 
as  the  development  of  command  training  program  objectives.  The  design 
of  instruction;  progranming  training;  unit  training  directives;  and 
the  development  and  conduct  of  instruction.  The  latter  to  include  the 
selection/development  of  learning  objectives.  The  selection  of  in¬ 
structional  strategies,  and  the  evaluation  of  instructional  results. 
FUTURE 

For  now,  we  have  our  "Systems  Approach"  well  on  the  road  in  all 
our  enlisted  formal  schools  both  via  our  "Table  Top"  individual 
school  programs,  and  via  our  Gentras  system.  Unit  level  training, 
too,  is  now  under  the  umbrella.  Our  next  step  is  a  revision  to  our 
"Design  of  courses  of  instruction"  order  to  embrace  all  Marine  Corps 
Training/Educat ion  -  officer  and  enlisted,  technical  and  professional. 
This  we  are  doing  now,  in  conjunction  with  the  development  of  a 
supporting  Marine  Corps  Instructor's  Guide.  Hopefully,  by  July  of 
1972,  all  Marine  Corp  Training  will  be  under  the  purview  of  comp¬ 
lementary  Marine  Corp  directive  based  on  the  overall  principles  of 
the  systems  approach  to  training. 
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NAVAL  AIR  TRAININGS  PILOT  SYSTEM 


1 


Dr.  Charles  B.  Haven 
Naval  Air  Station 
Pensacola,  Florida 


The  word  "Pilot”  in  the  title  of  this  paper  is  an  acronym.  It 
does  not  mean  an  individual  who  operates  aircraft.  The  acronym 
Pilot  stands  for 

Performance-centered 
Individual ized 
Learner-Oriented 
Training 

Thus  the  Pilot  System  is  a  performance-centered,  individualized, 
learner-oriented  training  system. 

Performance-centered  means  to  us  in  Naval  Air  Training  that  the 
object  of  ail  instruction  is  performance.  Students  are  taught  to  do 
things  or  to  behave  in  a  prescribed  manner. 

Individualized  has  a  two-fold  meaning.  At  the  present  time  it 
means  that  students  do  not  receive  instruction  in  regulated  groups, 
all  studying  the  same  thing  at  the  same  time  in  lock-step  fashion. 

By  individualizing  the  instruction  students  can  move  independently, 
and  "soft-scheduling"  of  instruction  is  a  reality.  Rigid  patterns 
of  group  instruction  can  be  dispensed  with,  and  training  aircraft, 
simulators,  and  buildings  can  be  utilized  on  an  even-flow  basis. 

At  a  later  date,  individualized  can  mean  a  variable  curriculum  with 
individual  tracking  —  in  exemption,  validation,  and  remedial  work 
as  required  by  individual  needs. 

Learner -Oriented  means  that  all  instruction  is  addressed  to  and 
developed  for  the  learner,  and  the  principles  of  controlled  self- 
discovery  revealed  in  incremental  fashion  operates  insofar  as  possible 
to  let  the  learner  teach  himself.  The  learner  is  the  principal 
participant.  He  responds  and  reacts  continually  and  is  continuously 
involved  throughout  the  learning  session. 

Train ing  is  the  "T"  in  the  acronym  and  the  primary  reason  for 
our  existence  in  the  Naval  Air  Training  Command.  Our  job  is  to 
train  pilots  and  flight  officers  for  the  fleets  of  the  Navy. 

The  PILOT  System  is  thus  Naval  Air  Training* s  cut  at  an  engineered 
training  system,  developed  along  the  lines  of  the  systems  approach 
applied  to  the  design  of  a  total  instructional  system.1 
Origin  and  Development 

“  The  beginnings  of  the  systems  approach  to  training  now  identified 
as  the  PILOT  System  lay,  for  Naval  Air  Training,  in  the  programed  in¬ 
struction  movement  that  gained  impetus  in  the  Spring  of  1964  in  Naval 
Air  Training.  From  a  meagre  beginning  of  four  programed  textbooks 
developed  in-house  the  programed  instruction  movement  developed  in 
succeeding  months  to  an  inventory  of  135  or  more  (perhaps  1100-1200 
clock  hours  of  learning  time),  and  numerous  cornpar isons  were  made 
between  programed- textbook  instruction  and  conventional-lecture  dis¬ 
cussion  methods. 

Because  a  methodology  for  making  comparisons  was  fuzzy  and  lacking 
in  general,  an  associate  and  I  authored  for  the  Conmand  in  1967  a 
manual  entitled  Measuring  the  Effectiveness  of  Programed  Instruction2 
that  suggested  various  ways  or  procedures. 


1.  Reference  1 
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With  this  tool,  we  made  dozens  of  comparisons.  The  conclusions 
were  predominately  the  same;  the  lecture-discussion  method  would 
yield  for  us  about  a  60-65%  correct  on  a  post-instructional  test. 

A  good  programmed  text  on  precisely  the  same  content  would  raise  the 
mean  post-test  score  to  a  level  of  90%  in  about  two  thirds  of  the 
time. 

But,  despite  the  obvious  fact  that  on  identical  material  students 
earned  significantly  higher  post-test  scores  when  they  learned  with 
programmed  materials,  our  aviation  instructors  still  preferred,  per¬ 
haps  with  some  justification,  the  traditional  position  of  influence 
in  the  front  of  the  classroom.  The  programmed  textbooks  were  thus  not 
extensively  used  as  the  primary  means  of  instruction  to  replace  the 
lecture-discussion  method. 

In  the  interim,  however,  the  seeds  of  discovery  had  sprouted.  It 
had  been  perceived  by  many  by  this  time  that  a  well  wrought  programmed- 
learning  sequence  is  in  reality  an  incremental  learning  system  in 
miniature.  It  contained  like  Van  Leeuenhoek*s  drop  of  water  the 
essentials  of  a  viable  system.  It  contains  (1)  precisely  defined 
behavioristic  objectives  (2)  a  practial,  operable  instructional  method- 
logy  (3)  a  list  of  criterion  performance  measures  which  obviously 
relate  to  the  original  behavioristic  objectives.  Movement  from  the 
programed-instruction  concept  to  attempting  the  design  of  an  engineered 
total  instructional  system  was  thus  natural  and  perhaps  inevitable. 

In  1967  a  team  of  four  under  my  sponsorship  published  for  the 
Command  and  eye-opening  manual  entitled  Introduction  to  the  System 
Approach.1  This  approach  concepts  to  the  Naval  Air  Training  problem. 
But  it  served  mightily  in  broadening  the  views  of  line  aviators  and 
training  administrators  to  the  possibilities  and  potentials  of  the 
systems  approach. 

The  concept  envisioned  in  this  publication  was  audacious,  chiefly 
because  it  suggested  a  backwards  approach,  so  to  speak,  to  the  systems 
training  concept.  Actually,  it  suggested  that  a  systems  approach  to 
training  could  be  implemented  at  the  lowest  level  of  the  Training 
program  and  grow  upwards,  so  to  speak.  It  suggested  that  any  small 
segment  of  instruction  could  become  a  micro-system  of  learning  that 
could  participate,  as  it  were,  in  the  larger  or  macro-system;  and 
that  the  classic  approach  to  systems  training  could  be  obviated. 

It  appeared  impossible  at  that  time  to  attack  the  problem  of 
over-all  systems  design  for  naval  aviation,  since  at  that  time 
our  Staff  was  at  the  lowest  level  of  Naval  Air  Training.  With  the 
resent  realignment  of  flag  billets  and  Command  combinations,  our 
Staff  has  been  elevated  to  senior  status  and  control  of  all  Naval 
Air  Training.  But  a  classic  approach  to  systems  training-  that 
is,  analyzing  the  tasks  performed  by  the  graduate  of  the  training 
program  and  working  backward  to  construct  a  training  program  in¬ 
culcating  the  desired  qualities  --  wa-s  not  feasible  anyway.  Thus 
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our  original  assumption,  naive  as  it  might  have  been,  was  followed. 

We  preferred  to  believe  that  there  had  to  be  some  good  qualities  in 
our  60-year-old  aviation  training  program;  and  that  we  could  not  shut 
down  the  old  pipeline  while  we  were  designing  and  implementing  a 
bright,  shiny  new  one.  Hence  the  PILOT  System  is  being  implemented 
from  the  bottom  to  the  top  and  from  the  front  to  the  back,  rather  than 
being  promulgated  from  the  top  to  the  bottom  and  the  back  to  the  front. 
Of  course,  either  of  these  positions  is  not  entirely  possible.  A 
system  must  be  engineered,  at  least  in  theory,  at  both  the  front  and 
the  back  and  the  top  and  the  bottom  at  the  same  time.  If  the  system 
is  indeed  well  engineered,  any  portion  of  it  could  be  implemented  at 
any  level. 

THE  OVER— ALL  STRUCTURE  OF  THE  PILOT  SYSTEM 

T*he  architectonics  of  the  PILOT  System  are  in  place  in  naval  air. 
The  over-all  shape  of  the  system  is  clear.  We  have,  working  along 
the  tree  of  logic,  subdivided  the  total  aviation  training  system  into 
✓  five  subsystems j 

1.  The  vehicle/platform 

2.  The  enviroment 

3 .  The  human 

4.  The  man-machine  system 

5.  The  weapons 

The  branch  below  the  subsystem  is  the  discipline,  a  collection  of 
courses  of  a  common  nature,  like  aerodynamics.  Below  the  discipline 
is  the  actual  course  of  instruction.  Courses  of  instruction  are  in 
turn  subdivided  into  instructional  units .  These  are  the  self-contained 
instructional  micro-systems  which  are  the  avenue  of  learning  for  the 
student. 

By  using  this  kind  of  tree  logic  for  systems  identification  and 
coupling  each  of  these  segments  with  an  appropriate  alpha-numeric 
designator  plus  groups  for  units  identifies,  instructional  time,  and 
method  of  instruction,  we  have  devised  an  operable  and  unique  alpha¬ 
numeric  code  designator  for  each  micro-unit  within  the  system.  This 
basic  code  structure  permits  positive  identification,  and  accountability 
on  both  a  manual  or  computer  controlled  records  keeping  system.  Our 
publication  entitled  "Alpha-Numeric  Code  for  the  PILOT  System?'  * 
facilitates  the  encoding  process. 

The  instructional  content  of  the  PILOT  System  is  controlled  by 
the  same  tree  logic.  The  total  system  is  governed,  for  example, 
by  a  set  of  over-all  goals  which  of  course  are  related  to  mission. 

The  mission  is  prescribed  by  higher  authority,  but  goals  may  be  set 
by  the  commander  of  the  system.  Each  sub-system  has  a  segment  of  the 
over-all  goals  of  fulfill,  as  do  disciplines.  Courses  of  instruction 
within  the  discipline  must  have  educational  objectives.  Within  the 
course  of  instruction  are  the  instructional  units  which  must  assuredly 
be  comprised  of  specific  instructional  objectives.  Since  we  are 
building  our  PILOT  System  in  detail  from  the  bottom  strata  to  the  top, 
we  have  thus  been  most  concerned  with  the  specific  instructional  ob¬ 
jectives. 
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SPECIFIC  OBJECTIVES  AMD  CURRICULAR  VALIDITY 

Our  scheme  for  stating  specific  instructional  objectives  for  units 
from  the  start  has  been  to  couch  them  in  the  language  of  Bloom's 
Taxonomy  of  Educational  Objectives.  We  sort  the  objectives  into  the 
three  domains  —  the  cognitive),  the  affective,  and  the  psychomotor. 

This  scheme  gives  the  instructor  a  plan  of  attack,  so  to  speak  only 
an  expert  can  eat  a  pizza  without  first  slicing  it.  Each  objective 
is  neatly  written  on  a  special  form  provided  for  that  purpose.  On 

this  form  is  also  a  space  to  record  the  action  of  the  friendly 

"murder  board,"  a  group  of  instructors  and  senior  administrators 
specially  constituted  for  this  purpose.  On  an  attached  form  is  the 
criterion  measure  associated  with  and  developed  from  the  specific 
objective.  The  murder  board  acts  not  only  upon  the  specific  but  also 
upon  its  related  criterion  item.  If  the  "murder  board"  should  approve 
of  the  objective  and  the  item,  it  will  then  be  included  with  other  valid 
objectives  of  the  unit.  The  criterion  measure,  written  in  triplicate, 
is  then  introduced  into  the  testing  sequence  for  the  item,  course, 
and  phase.  If  the  murder  board  should  not  approve  the  objective  or 
the  item,  both  are  junked  and  the  unit  writer  returns  to  his  work¬ 
bench  for  another  one.  These  procedures  and  specimen  copies  of  the 

software  forms  are  published  in  our  manual  extitled  "PILOT  System 
Testing  SoftwareMl(1971). 

The  procedures  described  above  may  impress  the  audience  with  its 
naivete  inasmuch  as  it  makes  no  claim  to  statistical  vilidity.  It 
does,  however,  function  quite  well,  and  its  quality  of  curricular 
validity  is  perhaps  the  highest  attainable.  A  greal  deal  of  "crud" 
has  been  purged  from  the  system  by  means  of  these  murder  boards. 

The  murder  boards  have  helped  to  keep  command  goals,  course 
educational  objectives,  and  specific  instructional  objectives  of  units 
in  closer  alignment  than  ever  before. 

PREPARING  FOR  INSTRUCTION 

With  his  list  approved  instructional  objectives  in  hand,  the 
instructor— technic ian  sets  to  work  hammering  out  his  PILOT  System 
unit.  This  is  a  bookley,when  finished,  that  looks  like  this.  The 
format  for  the  unit  is  prescribed  by  our  publication  entitled 
"Model  Unit  for  the  PILOT  System,  Instructional  Software" 2( 1971) . 

The  unit  contains  a  pre-test,  a  list  of  objectives,  a  list  of 
materials  needed  to  complete  the  study,  a  programed  or  structured 
learning  sequence,  and  a  unit  post-test,  with  complete  directions  and 
instructions  provided  for  the  learner. 

Audio-visual  support  of  the  PILOT  System  at  the  present  time  is 
our  toughest  problem.  We  are  limited  almost  exclusively  to  group 
showings  of  conventional  training  films  in  16mm  format.  These  are 
difficult  to  blend  effectively  with  our  individualized  performance- 
centered  system.  Group  showings  are  arranged,  however,  during  the 
band  of  the  training  day.  Providing  adequate  interaction  with  films 
utilized  in  this  fashion  is  impossible,  of  course,  except  in  actuality. 
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Our  long-range  solution  to  the  audio-visual  problem  is  to  pro¬ 
vide  individualized  instructional  TV  cassette  format  single-concept 
television  tapes  cut  precisely  to  the  specifications  of  the  instruc¬ 
tional  unit  for  which  they  will  be  made.  The  student  would  draw  the 
appropriate  instructional  TV  cassette  with  the  instructional  unit. 

He  would  go  to  the  cassette  players  located  at  various  sport  in  the 
learning  areas,  place  the  cassette  in  the  player  as  instructed,  and 
the  player  would  make  its  presentation  through  an  ordinary  11”  color 
television  set.  Listening  would  be  provided  through  an  earplug.  We 
envision  one  cassette-player/instructional  TV  set  available  for  every 
three  students  under  instruction.  Perhaps  two-thirds  of  the  approx¬ 
imately  2,500  PILOT  System  units  in  the  system  will  require  the 
dynamic  qualities  that  only  a  combination  of  motion  and  sound  can 
provide. 

The  log-jam  in  the  instructional  TV  cassette  production  business 
may  be  about  to  break  up.  According  to  Popular  Science  (Oct  1971 )* 
both  Sears,  Roebuck  &  Company  and  Montgomery  Ward  will  feature  an 
instructional  TV  Tape  player  for  home  television  sets.  The  instruc¬ 
tional  TV  cassette  manufactured  by  Cartivision  will  be  backed  up  by 
a  library  of  850  tapes.  The  whole  package  is  made  by  AVCO. 

The  central  instructional  TV  recording  facility  now  in  existence 
under  the  control  of  the  Chief  of  Naval  Air  Training  will  be  expanded 
to  service  the  instructional  TV  production  requirements  of  the  PILOT 
System.  This  central  producing  studio  and  staff  would  be  capable  of 
recording  and  distributing  perhaps  20  to  20  single-concept  instructional 
TV  taped  programs  per  week.  The  primary  source  of  programs  content 
would  be  the  instructor-managers  of  various  instructional  units  in 
the  PILOT  System. 

Already  under  development  for  the  instructors  use  in  presenting 
original  content  for  instructional  TV  tapes  is  a  workbook-1 ike-publi- 
cation  for  him  to  fill  in  to  start  his  instructional  TV  taped  program 
on  its  way  to  production.  The  workbook  will  be  designed  so  that  the 
instructor  can  relate  his  material  in  logical  fashion  to  his  specific 
instructional  objectives  and  present  it  in  story-board  fashion.  He 
would  rough  in  the  visual  content  in  frames  provided  and  would  supply 
the  narration  in  spaces  associated  with  the  video.  We  believe  this 
workbook  will  provide  the  central  recording  studio  the  essential 
ingredients  for  completing  the  production  and  delivering  the  pack¬ 
aged  cassettes  within  a  few  days. 

EARLY  RESULTS 

The  performance  of  the  PILOT  System  is  fully  implemented  courses 
has  been  good  to  excellent.  The  first  course  totally  converted  was 
Visual  Flight  Rules  Navigation  (Basic  Prop),  formerly  a  30-hour  lecture- 
discussion  course.  Under  the""PIIJ'o¥  'System  the  course  as  converted  to 
17  instructional  units.  In  June,  1971,  115  students  proceeded  through 
this  course  with  mean-post-test  performance  of  84.62  per  cent  correct 
in  mean  time  of  28  hours  and  45  minutes. 
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Also  in  June  1971  in  our  Basic  Phase  (JET)  Instrument  Flight 
Rules  Navigation  course,  82  students  received  a  mean  post-test 
mark  of  7672  per  cent  correct,/  This  course  contains  20  instructional 
units,  whose  content  equates  to  22  hours  of  lecture-discussion 
material.  Instructional  time  under  the  PILOT  format  was  approx¬ 
imately  the  same. 

Both  of  the  courses  just  cited  are  being  extensively  revised  on 
a  unit-by-unit  basis,  using  the  pre-  and  post-test  data  collected  to 
bolster  the  weaker  portions  of  the  instructional  sequence. 

Better  results  have  since  been  obtained  in  our  Primary  Phase 
courses.  From  16  August  to  16  September  1971  approximately  155 
students  naval  aviators  progressed  through  three  courses  in  this 
phase  that  had  been  converted  to  and  instructed  in  the  PILOT  format. 

The  results  in  brief  review  were  reported  as  follows:* 

CONVENTIONAL  PILOT 

LECTURE-  INSTRUCTIONAL  PRE-TEST  POST-TEST 

COURSE  DISCUSSION  TIME _ MEAN  MEAN 

Flight  Rules  and  Regulations  4  ~  3.5  89.66% 

T-34  Engineering  Systems  13  9.3  45.15%  92.76% 

T-34  Aerodynamics  9  6.1  38.37%  91.50% 

*  Memorandum  of  Training  Officer,  NAS  Saufley  Field  of  9-16-71 

On  the  strength  of  these  kinds  of  results,  a  dozen  other  courses 
of  varying  length  are  being  readied  in  the  PILOT  format  for  imple¬ 
mentation  in  the  immediate  future. 

SUMMATION 

In  summation  of  comments  on  Naval  Air  PILOT  System,  the  following 
points  deserve  mention: 

1.  The  system  itself  was  designed  from  the  bottom-up.  There  was  no 
classical  approach  to  it.  No  task-analysis  effort  except  at  the  course 
and  unit  level. 

2.  There  is  no  electronic  hardware  associated  with  the  system  at  the 
present  time  except  for  a  trickle  of  instructional  TV.  There  are  no 
response-system  mechanisms  or  information  retrieval  provisions.  We 
are  strenuously  pushing  for  an  individualized  instructional  TV  cassette 
distribution  and  have  good  reason  to  believe  we  will  be  so  funded  in 
the  near  future. 

3.  The  software  exists  predominately  in  printed  booklet  form  and  is 
generated  in-house  by  regularly  assigned  academic  and  flight  support 
instructors. 

4.  The  results  of  the  first  five  courses  administered  under  the  PILOT 
System  format  range  from  good  to  excellent. 

5.  Training  administrators  are  afforded  the  best  view  of  the  training 
process  they  have  ever  had  before.  The  low-performance  areas  in  the 
system  are  readily  identifiable. 

6.  The  PILOT  System  requires  a  lot  of  hard  work  and  great  attention 
to  detail,  but  the  pay-off  is  good.  It  works. 
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It  is  a  pleasure  to  again  present  an  overview  of  the  current 
status  of  the  application  of  the  "Systems  Approach"  to  Air  Force 
training  and  education  programs.  Needless  to  say,  much  has  happ¬ 
ened  in  Air  Force  training  since  the  last  CONARC  Training  Con¬ 
ference  at  Fort  Banning  in  September  1968.  The  Systems  Approach 
to  Training  was  then  being  applied  only  to  Air  Force  technical 
training.  Now  it  has  been  extended  to  apply  to  all  Air  Force 
Training  and  education  programs.  Our  basic  handbooks,  Air  Force 
Manual  50-2,  Instructional  System  Development,  was  published  in 
December  1970.  Flying  Training  and  Professional  Military  Education 
are  now  included  under  the  systems  approach  planning  umbrella. 

Through  a  marriage  of  personnel  and  training  research  and  weapon 
system  planning,  the  Air  Force  System  Approach  to  Training  has 
emerged. 

Let  us  look  at  the  Air  Force  training  system  so  you  will  better 
understand  the  enviroment  in  which  we  live  and  work.  Note  that  the 
Air  Training  Command,  my  parent  command,  it  charged  with  the  res¬ 
ponsibility  of  recruiting  and  training  the  majority  of  Air  Force 
personnel.  However,  keep  in  mind  that  the  Air  Force  Academy  and  the 
Air  University  are  primarily  responsible  for  professional  military 
education.  Note  also  that  Air  Training  Command  carries  out  its 
responsibilities  through  flying  training  schools,  four  technical 
training  centers,  the  USAF  Recruiting  Service,  and  one  basic  military 
training  center.  Note  the  location  of  Sheppard  Air  Force  Base  Tech¬ 
nical  Training  Center,  which  has  a  key  role  in  the  Air  Force  Systems 
Approach  to  Training.  First  of  all,  our  Training  Center  at  Sheppard 
Air  Force  Base  has  the  primary  curriculum  responsibility  for  the 
separate  Air  Force  Training  and  Education  Career  Field  and  supporting 
resident  and  correspondence  type  courses  and  manuals.  We  are  also 
responsible  for  development  of  the  Air  Force  Manual  on  the  Systems 
Approach  to  Training,  which  as  mentioned  above  is  Air  Force  Manual 
50-2,  Instructional  System  Development.  Further,  we  are  responsible 
for  conducting  several  Air  Force  courses  in  this  new  technology  of 
instruction  and  curriculum  development.  Let  us  now  review  some  of 
our  experience  and  problems  in  this  pioneering  effort. 

Personnel  and  training  research  efforts  from  industry.  Department 
of  Defense  agencies,  the  US  Office  of  Education  and  other  public  and 
private  sources  have  made  great  contributions  to  our  Air  Force 
program.  However,  since  the  early  1950s  the  Air  Force  has  been  using 
the  concept  of  the  weapon  system,  its  subsystems  and  components.  The 
human  component  must  be  considered  at  every  stage  of  weapon  system 
development,  from  initial  conception  to  obsolescence  of  the  weapon 
system— a  "cradle  to  the  grave  concept."  In  1962,  Dr.  Robert  Gagne, 
who  is  our  consultant  in  this  CONARC  system  engineering  workshop, 
brough  together  many  important  papers  in  the  systems  approach  in  the 
McGraw-Hill  publication,  "Psychological  Principles  in  Systems  Develop¬ 
ment."  This  classic  text  should  still  be  required  reading  in  this 
area.  The  Air  Force  Human  Research  Laboratories  have  also  made  great 
contributions  to  our  efforts.  We  have  drawn  heavily  from  research 
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sponsored  by  the  US  Office  of  Education  and  from  both  the  US  Army 
and  US  Navy  research  projects  in  this  area.  Even  with  all  this, 

1  think  it  is  obvious  that  we  are  putting  into  practice  only  a 
small  portion  of  what  personnel  and  training  research  indicates  is 
the  best  of  training  technology.  Ve  are  in  the  same  position  as 
the  farmer  who  says  "I  ain*t  farming  half  as  good  as  I  know  how." 

Application  of  the  "systems"  approach  to  Air  Force  training 
programs  has  been  stimulating  and  has  forced  us  into  new  creative 
planning  efforts.  New  training  programs  are  being  developed  and 
existing  courses  revitalized  through  application  of  this  approach. 

The  weapon  system  designer  begins  with  an  operational  requirement— 
a  statement  of  overall  objectives  to  be  achieved— and  produces  an 
arrangement  of  subsystems  which  will  fulfill  the  operational  ob¬ 
jectives.  A  series  of  tests  is  then  conducted  to  insure  that  the 
system  does,  in  fact,  fulfill  the  requirement.  Let*s  look  at  an 
illustration  of  the  events  in  a  weapon  system  development  cycle: 
Definition  of  System  Objective 
Advanced  Design 

Assignment  of  Functions  to  Man  and  Machines 

Task  Description 

Task  Analysis 

Job  Design 

Job  Aids 

Personnel  Selection 
Individual  Training 
Training  Devices  or  Media 
Performance  Measures 
Team  Training 
System  Training 
System  Evaluation 
System  Operational  Stage 

The  development  of  a  training  program  under  the  systems  approach 
proceeds  in  a  manner  similar  to  the  development  cycle  of  a  major  wea¬ 
pon  system..  The  Air  Force  "Systems  Approach  to  Training,"  as  explained 
in  AFM  50-2,  consists  of  the  following  interlocking  steps: 

Analyze  System  Requirements 

Define  Education  or  Training  Requirements 

Develop  Objectives  and  Tests 

Plan,  Develop,  and  Validate  Instruction 

Conduct  and  Evaluate  Instruction 

Before  we  go  into  a  discussion  of  the  Air  Force  Systems  Approach 
to  Training,  let  us  look  at  some  other  models  for  curriculum  develop¬ 
ment  which  influenced  our  thinking.  An  excellent  training  program 
development  model  was  proposed  as  a  result  of  a  Department  of  Defense 
directed  study  of  curriculun  content.  This  study  was  conducted  by  the 
Hunan  Resources  Research  Office  (HumRRO)  and  consisted  of  the  follow¬ 
ing  seven  steps: 

Conduct  System  Analysis 
Develop  Task  Inventory 
Develop  Job  Model 
Conduct  Task  Analysis 
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Derive  Training  Objectives 
Develop  Training  Program 

Monitor  Trained  Product  and  Modify  Curriculum  as  Required 

One  other  system  development  model  to  which  1  oall  your  attention 
for  comparison  with  the  Air  Force  model  is  that  specified  in  your 
COHARC  Regulation  350-100-1,  which  consists  of  the  following  steps: 

Job  Analysis 

Selecting  Tasks  for  School  Training 
Training  Analysis 
Developing  Training  Materials 
Developing  Testing  Materials 
Conduct  of  Training 
Quality  Control 

This  Army  system  engineering  (course  design)  model  also  indlueneed 
our  Air  Force  development.  Thus  we  see  in  all  these  different  models 
much  similarity,  yet  varying  emphasis  on  different  steps.  All  of  them 
reflect  different  interpretations  or  applications  of  the  systems  app¬ 
roach  to  course  development. 

The  Air  Force  Systems  Approach  to  Training  suggests  a  rigid  step- 
by-step  system,  however,  there  will  always  be  variations  in  the  order 
or  number  of  steps,  depending  upon  the  specific  situation  and  the  nature 
or  complexity  of  the  training  involved.  There  is  much  interaction 
between  steps.  It  may  be  necessary  to  backtrack  and  reacccmplish  or 
modify  an  earlier  step  when  it  is  discovered  that  it  does  not  provide 
the  necessary  input  for  a  later  step.  The  manner  of  application  and 
the  degree  of  detailed  development  can  be  expected  to  differ;  however, 
the  underlying  principles  of  this  systems  approach  will  remain  un¬ 
changed.  A  brief  description  of  these  steps  follows; 

Step  1  -  ANALYZE  SYSTEM  REQUIREMENTS 

There  are  obivously  many  factors  in  an  operational  system  which 
must  be  considered.  An  understanding  of  the  system  within  which  each 
job  is  to  be  performed  is  necessary  including  the  operational  or  work¬ 
ing  environment,  the  missions  of  the  system  of  which  the  job  is  a 
part,  and  the  functions  of  the  system  components.  The  man  and  machine 
functions  and  interface  or  interrelationships  of  the  various  system 
are  analyzed.  Important  sources  of  information  for  training  need  are 
the  weapon  or  support  system  job  performance  requirements.  Properly 
developed  statements  of  job  performance  requirements  describe  the 
operational  context,  quantitative  and  qualitative  limits,  and  critical 
factors  of  job  performance.  In  the  analysis  of  job  tasks,  it  may  be 
necessary  to  propose  reallocation  of  tasks  to  different  job  specialties 
or  even  to  combine  certain  job  specialties  on  a  particular  weapon 
system  or  support  system.  System  analysis  places  the  job  toward  which 
training  is  to  be  designed  in  the  proper  perspective  or  the  objectives, 
requirements  and  environment  of  the  operational  system.  This  step 
provides  the  initial  documentation  of  job  performance  requirements 
which  forma  the  basis  for  all  training  course  development. 
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Step  2-  DETERMINE  EDUCATION  OR  TRAINING  REQUIREMENTS 

fKis  step  represents  a  major  decision  in  training  program 
development.  When  the  job  performance  requirements  are  known, 
it  is  necessary  to  determine  which  requirements  will  be  met 
through  personnel  reassignments  of  already  qualified  individuals, 
through  formal  school  training,  or  through  on-the-job  training. 

The  most  efficient  and  cost  effective  method  of  meeting  the  re¬ 
quirement  will  be  selected.  The  entire  skill  level  progression 
and  on-the-job  training  and  formal  school  requirements  are  con¬ 
sidered  at  this  time.  Training  standards  are  published  and  be¬ 
come  the  guide  for  resident  school  planning,  correspondence  courses 
and  On  The  Job  Training.  It  should  be  emphasized  that  the  Air 
Force  Specialty  Training  Standard  (AFR  50-34)  outlines  the  training 
required  to  achieve  various  skill  levels  with  job  specialties. 

Through  its  use,  individual  training  (both  formal  school  and  on-the- 
job  training)  is  standarized  and  the  quality  of  training  is  con¬ 
trolled. 

Step  3  -  DEVELOP  OBJECTIVES  AND  TESTS 

The  list  of  tasks,  knowledges,  and  attitudes  provided  through 
the  previous  step  will  be  carefully  reviewed  to  determine  currency, 
accuracy,  and  adequacy.  Criterion  objectives  must  be  carefully 
determined  from  a  detailed  analysis  of  the  job  or  task  listing  in 
the  Training  Standard.  Criterion  objectives  should  describe  a  job- 
related  performance  and  the  conditions  under  which  it  should  be 
observed  or  measured.  Because  the  learning  objectives  are  based  on 
a  detailed  analysis  of  the  job  for  which  the  students  are  to  be 
trained,  only  essential  and  relevant  objectives  are  included.  Once 
criterion  objectives  objectives  have  been  developed,  the  enabling 
objectives  (or  intermediate  knowledge,  skills  or  attitudes)  required 
to  reach  the  terminal,  criterion  objectives  are  identified.  All 
objectives  then  are  essential,  and  the  proficiency  measurement  tests 
or  device  should  throughly  measure  each  and  every  objective. 

Step  4  -  PLAN.  DEVELOP  AMD  VALIDATE  INSTRUCTION 

Using  the  objectives  developed  in  the  previous  step,  the  most 
appropriate  teaching-learning  activity  should  be  planned  for  each  of 
these  objectives,  based  on  principles  or  learning.  This  action  would 
include  a  determination  of  the  most  efficient  and  effective  teaching 
media  and  the  most  appropriate  supporting  training  equipment,  mater¬ 
ials  and  other  audiovisual  aids.  The  course  teaching-learning,  activities, 
should  then  be  sequenced  for  the  best  learning  relationshio  and  use 
of  resources  (time,  facilities  and  equipment.)  In  general,  it  means 
that,  when  possible,  the  learner  moves  from  the  simple  to  the  complex, 
from  the  known  to  the  unknown,  from  the  concrete  to  the  abstract. 

During  this  step,  programmed  texts,  study  guides,  workbooks,  in¬ 
structor  guides  and  lesson  plans  are  developed.  Whether  the  selected 
activity  is  a  lecture,  film  strip,  a  tape,  a  series  of  slides,  a 
performance  project,  a  programmed  test,  a  script  for  a  movie  or  a 
combination  of  several  of  these,  the  planner  knows  that  the  activity 
must  prepare  the  student  to  gain  specific  knowledge 
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and  skills.  Concurrently  with  the  development  of  training  materials, 
other  actions  will  be  taking  place.  Facilities  will  need  to  be  ob¬ 
tained  and/or  modified  to  accommodate  student  and  trainer  installa¬ 
tion.  Trainers  need  to  be  built  and/or  installed,  and  supplies  will 
have  to  be  procured.  Instructors  will  also  have  to  be  obtained  and 
trained.  The  validation  process  will  also  take  place  during  this 
step.  After  a  system  has  been  developed  and  validated,  the  system 
is  ready  for  operational  use.  Course  materials  should  continue  to 
be  revised  until  the  acceptable  standard  is  reached. 

one  in  terms  of  the  continuing 
success  of  a  course.  Primarily  a  course  is  evaluated  in  terms  of 
student  attainment  or  non-attainment  of  the  stated  objectives.  The 
evaluation  process  includes  both  internal  and  external  procedures  such 
as  student  grades,  performance  tests,  student  critiques,  and  super¬ 
visor’s  opinions  obtained  by  interviews  and  questionnaire  surveys. 

Thus  a  training  program  is  in  a  constant,  continuing  revision  and 
updating  process. 

APPLICATIONS  OF  THE  SYSTEMS  APPROACH 

The  Air  Force  Systems  Approach  to  training  is  now  being  applied 
to  all  types  of  courses — technical,  flying  and  professional  military 
education.  Some  studies  of  existing  technical  courses  under  the 
systems  approach  have  been  in  depth,  while  others  have  been  only 
token  efforts.  Many  courses  have  been  reduced  in  length  as  a  result. 
Others  have  been  increased  in  length,  so  do  not  automatically  expect 
to  reduce  courses.  Greatest  savings  are  expected  through  reduction  in 
the  individual  student  trainee  time  required  to  complete  their 
assigned  courses.  Average  annual  savings  of  about  $1,000,000  per  year 
at  each  of  our  Air  Force  technical  training  centers  have  already  been 
achieved.  Progress  has  been  slower  than  anticipated,  and  it  is  ex¬ 
pected  that  at  least  another  three  years  will  be  required  to  complete 
the  review  of  all  technical  courses. 

While  the  major  emphasis  in  application  of  the  systems  approach 
has  been  in  technical  training  programs,  application  to  C-130E 
Aircrew  Transitional  Training  within  the  Tactical  Air  Command  achieved 
not  able  results.  Specific  objectives  were  derived  from  a  task 
analysis  of  the  aircrew  members'  jobs.  The  training  program  was  pre¬ 
pared  to  develop  proficiency  in  the  specific  duties  required.  Much 
greater  use  was  made  of  self-instructional,  self-pacing  materials, 
including  a  variety  of  audiovisuals.  The  training  program  was  con¬ 
ducted,  evaluated  and  revised  over  a  six-month  period.  Results  in¬ 
cluded  the  following:  (1)  Classroom  instruction  was  reduced  about 
50%  (2)  flying  hours  were  reduced  from  45  to  35  hours;  (3)  length 
of  training  was  reduced  37%  per  trainee;  (4)  and  verified  annual 
savings  of  about  five  million  dollars  were  realized. 

Other  Air  Force  flying  training  program  in  which  the  systems 
approach  is  being  applied  in  depth  include  the  A-7D  aircrew  training 
programs  and  both  the  undergraduate  pilot  and  navigator  programs.  Pro¬ 
gress  to  date  indicates  that  a  reduction  in  flying  hours  will  be 
possible,  with  increased  amounts  of  flight  simulator  time  and  greater 
usage  of  multimedia  and  self-pacing  materials. 


5  -CONDUCT  AND  EVALUATE  INSTRUCTION 
The  final  step  is  the  most  important 


Problems  encountered  included  lack  of  understanding  of  the  systems 


roach  an 
ort  requ 


d  concepts,  terminology,  and  misconceptions  as  to  time  and 
ired  to  implement  the  new  procedures.  Training  programs 
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for  various  levels  of  management  have  been  sent  to  major  Air  Force 
installations  to  teach  the  systems  approach  to  training.  Training 
directives  are  being  revised  to  reflect  the  systems  approach.  All 
new  training  course  curriculum  documents  must  indicate  whether  or 
not  the  systems  approach  has  been  used.  Additional  guidelines  and 
revisions  to  AFM  50-2,  Instructional  System  Development,  are  now 
under  consideration. 

The  benefits  from  the  Air  Force  systems  approach  are  many,  The 
student  has  a  better  knowledge  of  what  is  expected  of  him  and  why. 

The  instructor  has  a  better  knowledge  of  training  goals  and  objectives. 
Supervisors  and  administrators  have  a  better  "yardstick"  or  training 
and  educational  program  effectiveness.  The  users  of  our  graduates  can 
also  expect  better  qualified  graduates  at  reduced  costs.  It  has  been 
a  real  pleasure  for  me  to  share  these  Air  Force  experiences  with  such 
a  dedicated  group  of  training  managers  and  executives  with  similar 
problems.  I'm  sure  that  we  in  the  Air  Force  will  benefit  greatly  from 
this  exchange  of  ideas  on  systems  engineering  through  this  CONARC 
training  workshop.  May  I  compliment  you  on  the  progress  you  have 
already  achieved. 
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SYSTEMS  APPROACH  TO  COURSE  DESIGN  IN  THE  CIVILIAN  COMMUNITY 


Dr.  Robert  M.  Gagne' 

Florida  State  University 
Tallahassee,  Florida  2f>306 


It's  a  pleasure  to  be  with  you  and  hear  the  various  present¬ 
ations  in  this  session.  There  is  a  hand  out,  that's  being  dis¬ 
tributed  which  sort  of  gives  an  outline  of  what  I  have  to  say 
about  system  development  in  the  civilian  sector.  As  I  have  list¬ 
ened  to  the  other  presentations  I  realize  that  all  of  them  indicate 
that  they  have  accepted  the  systems  language  and  the  ideas  of  systems 
engineering  and  that's  a  very  impressive  thing  to  me  to  hear  that. 

I  am  reminded  of  the  fact  that  one  shouldn't  think  that  there  are 
great  developments  out  there  in  the  civilian  world  that  are  necessarily 
greater  than  those  that  are  going  on  within  the  military  services. 

This  table  that  I  handed  out  to  you  has  a  column  on  the  left  which 
lists  the  stages  in  man  machine  system  design.  That's  just  sort  of 
for  reference  purposes.  Now  the  next  column  indicates  the  stages 
in  educational  system  development  and  uses  the  kinds  of  words  that 
are  common  in  that  sort  of  enterprise  at  least  they  are,  I  think, 
very  well  understood  when  system  development  is  carried  out  in  civil¬ 
ian  schools  and  courses  and  so  on.  Then,  in  the  final  column  I  have 
tried  to  list  the  kinds  of  techniques  that  are  available  to  carry 
out  educational  system  development  in  the  civilian  sector.  It  may  be 
of  interest  to  you. . .someone  asked  me  this  question  and  I  might  try 
to  answer  it  here  as  best  I  can.  Where  does  system  development  go 
on  in  the  schools  or  in  the  school  systems  of  the  civilian  sector? 

Well,  it  goes  on  in  a  lot  of  places.  It  may  go  on,  of  course,  in  an 
individual  school.  Teachers  of  this  school  may  decide  or  may  become 
interested  in  developing  a  particular  course  and  then  they  find  a 
way  to  get  enough  time,  through  the  cooperation  of  their  principal 
or  their  superintendent  to  devote  their  time  to  this  kind  of  develop¬ 
ment.  To  an  increasing  extent,  system  development  is  being  undertaken 
by  school  systems  and,  of  course  as  you  know,  there  is  an  enormous 
variation  in  the  size  of  school  systems  throughout  this  great  country 
of  ours  and  some  of  them  are  very  small  and  some  of  them  are  quite 
large  and  naturally  the  larger  ones  are  the  ones  that  usually  have  the 
resources  that  enable  them  to  undertake  system  development.  Then,  of 
course,  it  is  undertaken  by  support  from  the  Federal  government  in  a 
number  of  ways.  There  are,  as  you  know,  regional  laboratories  in  various 
parts  of  the  country  and  they  are  engaged  in  course  and  curriculum  and 
sometimes  ttotal  school  development.  They  may  undertake  the  undertaking 
to  develop  a  new  curriculum  in  elementary  mathematics  or  a  new  curriculum 
in  reading  and  many  other  examples  of  that  sort.  So  I’d  say,  that 
system  development  goes  on  in  many  places  in  the  civilian  sector  and 
it  exhibits  a  great  diversity  -  as  great  a  diversity  or  perhaps  more, 
than  you  have,  and  is  exhibited  in  the  kinds  of  presentations  I  have 
heard  in  this  session.  One  of  the  problems,  of  course,  in  that 
diversity  in  trying  to  select  out  of  it  what  is  best  and  trying  to  put 
into  effect  what  is  best  and  trying  to  get  it,  as  the  word  is,  disseminated. 
1  am  not  sure  that  "disseminated"  is  the  proper  concept  but  try  to  get 
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it  used  -  it*s  not  easy  because,  as  you  know,  between  and  among 
schools  there  is  no  command  authority.  They  all  have  their  own 
authority  and  so  they  all  have  their  own  ideas  and  it  is  a  very 
difficult  process  to  get  new  ideas,  even  well-developed  ideas. 

Accepted  throughout  a  large  segment  of  the  school  community 
in  the  civilian  sector.  Maybe  you  have  an  advantage  over  this, 

1  am  not  sure,  but  certainly  that  is  the  case.  You  know  you  are 
dealing  with  hundreds  of  individual  school  systems  and  they  just 
don*t  have  any  relation  to  each  other.  They  are  supposed  to  be 
locally  controlled  and  so  it*s  very  difficult  to  disseminate  new 
development  among  them.  Going  back,  then,  to  the  table,  I  have 
tried  to  list  here  in  the  final  column,  as  I  said,  a  number  of 
techniques  that  are  available  at  the  various  stages  of  educat¬ 
ional  development.  Now  this  might  be  applicable  to  development 
if  we  were  talking  about  a  single  school.  You  know  one  might 
want  to  say,  "Let  us  take  this  school,  which  is  not  a  very  good 
school,  and  make  it  into  a  really  good  one,"  or  "Let  us  take 
this  school  and  have  it  a  school  that  uses  individualized  in¬ 
struction  in  which  its  surriculum  is  designed  according  to 
system  principles."  That's  possible  -  not  that  there  are  many 
instances  of  that  sort  but  there  are  some.  Equally  well,  I 
think,  or  almost  equally  well  -  these  techniques  apply  to  the 
development  of  single  courses  or  perhaps  sets  of  courses  that 
form  a  curricula  such  as  those.  I'd  say,  such  as  a  curriculum 
in  elementary  science.  If  we  were  going  to  talk  about  system 
development  of  entire  school  systems,  we  would  have  to  deal 
with  a  somewhat  larger  context  that  would  include  aspects  of 
finance  and  management  of  school  systems  and  these  are  not  re¬ 
presented  here,  I  just  wanted  to  make  that  clear.  This  deal  with 
the  smaller  units  such  as  courses  curricula  or  at  the  most,  you 
know,  a  total  school  program.  Now,  these  techniques  that  are 
available,  I  think,  the  ones  that  are  listed  here  and  I  will  try 
to  say  a  word  about  them  as  I  go  on,  these  techniques  are  avail¬ 
able.  Some  of  them  are  old,  as  you  see.  Some  of  them  are  fairly 
new,  and  others  are  really  not  quite  developed  yet  even  but  they 
are  the  kind  of  things  that  seem  to  me  to  have  appeared  in  the 
various  literature  and  the  various  projects  and  other  things  that 
I  have  had  contact  with  that  go  on  in  system  development  so  far  as 
the  civilian  sector  is  concerned.  Often  times,  looking  at  the 
table  again,  one  wants  to  begin  a  development  activity  by  being 
concerned  with  the  systems  purpose.  Obviously  one  does  that  when 
he  is  talking  about  a  reference  system  or  something  of  that  sort. 
What  about  a  school  system?  You  know,  are  you  concerned  about 
that,  mmnmh  (affirmative),  you  do  that.  Now,  I  think  you  are,  you 
have  some  ways  of  getting  from  users  who  are  going  to  ultimately 
employ  the  trainees  some  idea  of  what  the  purposes  should  be.  I 
am  not  sure,  how  systematically  that's  done.  Sometimes  I  think 
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it's  done  quite  systematically  and  sometimes  less  so  in  your  sit¬ 
uation.  In  the  case  of  the  civilian  sector,  it  isn*t  often  done. 

One  depends  upon  writers  on  educational  philosophy  or  national 
commissions  of  various  sorts.  You  probably  have  all  heard  about 
something,  I  think,  called  the  seven  cardinal  principles  which  more 
formulated  by  a  national  commission  back  in  1919,  I  think  it  was. 

They  are  pretty  go. d.  You  know,  they  are  not  bad.  They— re  really 
quite  good.  The  language  sounds  a  little  obsolete  but  the  meaning 
is  not  bad.  And  there  have  been  various  attempts  like  that  by  ed¬ 
ucational  writers  coming  down  through  the  decades  up  to  the  present, 
and  you  find  people  like  John  Gardiner  writing  about  this  -  not  so 
many  years  ago.  So  there  is  a  dependence,  I  think,  upon  educational 
philosophy. 

There  is  another  way  to  do  it  and  that  is  to  try  to  involve  the 
parents  in  this  decision.  What  is  it  that  you  want  the  school  to  do? 

It's  not  a  bad  idea.  It  has  been  tried  in  a  number  of  places. 

I  think  that  it  has  to  be  done  rather  carefully,  because  in  a 
sense,  I  would  say,  you  donft  want  the  parent  to  get  involved  in  the 
details.  If  you  do  that  then  the  whole  thing  might  go  to  pieces.  But 
you  want  them  to  be  able  to  express  their  values  about  what  on  earth 
is  this  school  all  about,  anyway?  When  you  ask  parents  this  question 
with  respect,  for  example,  to  elementary  school,  the  studies  show 
that  the  thing  they  are  likely  to  put  at  the  top  on  the  list  is  the 
mastery  of  basic  skills;  reading,  writing  and  arithmetic.  Close  to 
that  would  be  a  kind  of  being-able-to-get-along-with-other -people  sort 
of  value  that  parents  hold.  These  kind  of  things  have  turned  up  again 
and  again  in  various  parts  of  the  country  and  it's  an  interesting 
fact.  So  that*s  one  way  to  go  about  this  business  of  finding  out  what 
the  goal  of  such  a  system  would  be.  It  hasn't  been  used  widely,  but 
it  has  been  done.  What  about  operations  design  or  system  design?  Well, 
I'  11  skip  over  these  rather  rapidly  here.  There  are,  of  course, 
standard  curricula  that  have  been  around  for  a  very  long  time,  and 
when  one  undertakes  to  make  a  new  design,  it  isn't  often  that  he  departs, 
really,  from  these  standard  curricula.  Now,  whether  that  is  good  or 
bad,  I'm  not  so  sure.  But  I  would  be  inclined,  1  think,  to  think  it's 
time,  perhaps,  that  we  looked  at  this  more  intensively.  Nevertheless, 
many  development  efforts  start  with  the  notion  there  is  this  standard 
curricula  for  working  with  an  elementary  school.  The  kids  are  going 
to  be  learning  reading  and  arithmetic  and  social  studies  and  science 
and  that's  it,  you  know,  it's  kind  of  rigid.  O.K.  Similarly,  one 
often  starts  with  ertain  accepted  school  management  procedures.  Now, 
there  are  some  new  ones  here  that  are  being  developed  also.  New  ways 
of  managing  schools  that,  perhaps,  take  into  account  the  new  ideas  of 
accountability  -  new  methods  of  personnel  management  which  might  take 
these  notions  into  account.  These  things  are  being  developed.  Let 
me  spend  a  little  time  on  system  design  in  terms  of  a  little  bit  more 
detail  here.  There  is  what  I  call  "output  analysis"  and  this  means 
analyzing  what  is  supposed  to  happen  in  terns  of  the  outcomes  of 
learning.  And  that  you  have  heard,  I  am  sure.  Other  speakers  have 
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spoken  of  various  schemes  for  doing  this.  It's  a  very  necessary 
part  of  system  development,  I  believe.  You  need  some  categories 
of  learning  outcomes.  Of  Course,  there  are  performance  objectives. 
That  is  the  next  category.  The  relation  between  these  two,  I 
think,  should  be  fairly  clear.  The  performance  objectives  may  be, 
you  know,  for  any  given  course  may  amount  to  hundreds  because  they 
are  all  individual  things  that  need  to  be  learned.  Output  analysis 
gives  you  some  kind  of  categorization  of  these  so  thay  you  can  deal 
with  them  in  terms  of  the  kinds  and  conditions  that  are  needed  for 
learning.  ••Learning  task  analysis"  -  well,  that  also,  I  find,  has 
been  mentioned  by  several  of  your  speakers  and  there  are  techniques 
for  doing  that.  There  is  also  something  which  1  call  here  "staffing 
analysis"  -  sometimes  called  "differentiated  staffing*’  -  and  there 
are  ways  of  analyzing  the  total  personnel  or  even  the  total  funds 
available  to  a  school,  and  at  the  same  time,  looking  at  the  kinds 
of  tasks  that  need  to  performed.  You  will  recognize  this  as,  you 
know,  a  task  analysis  kind  of  thing  that  lead  to  what  is  called 
"differentiated  staffing,"  which  means,  you  know,  not  just  the  teacher 
and  the  students  here  but  some  kind  of  aide  and  sub-professional  and 
other  kinds  of  clerical  activities  dif ferentizted  among  the  staff  of 
the  school.  A  number  of  schools  are  trying  out  developments  of  this 
sort. 

Well,  I  am  really  running  out  of  time  here  and  1  think  you  can 
follow  the  other  kinds  of  techniques  that  I  have  mentioned  here. 
Perhaps  one  of  the  things  that  I  would  emphasize  as  a  very  important 
development  which  I  know  is  very  relevant  to  what  you  are  doing  and 
one  which  you  use,  at  least  frequently  if  not  totally,  is  what  are 
called  here  "criterion  reference  tests"  and  "criterion  reference 
assessments."  This  is  the  kind  of  think  that  I  think  will  make  con¬ 
siderable  difference  in  the  ways  schools  operate.  If  they  are  able 
to  develop  instruments  or  ways  of  assessing  student  achievement  that 
are  truly  related  to  what  the  students  have  learned,  specifically  what 
they  have  learned,  you  have  these  things  in  the  form  of  performance 
tests  which  are,  usually  at  least,  criterion  reference  kinds  of  tests. 
I  think  I  see  a  great  deal  of  promise  in  this  kind  of  development  in 
the  civilian  sector  because,  it  seems  to  me,  it  will  finally  sort  of 
close  the  gap  between  what  the  teacher  is  doing  and  what  the  outcome 
is  and  can  be  observed  or  measured  to  be  as  far  as  the  student  is 
concerned.  So  -  O.K,  I  think  I  would  sum  up,  perhaps  by  saying,  you 
know,  there  are  alot  of  techniques  here  -  some  of  then  are  still  being 
worked  on.  I  am  sure  you  have  had  many  lists  and  I  can't  really  want 
to  impose  another  one  on  you.  I  would  simply  say  you  can  compare  many 
of  the  things  here  that  are  on  this  list  with  those  that  you  have 
heard  about  from  other  speakers.  Some  of  them,  perhaps  are  a  little 
different.  People  in  the  civilian  sector  continue  to  work  upon  the 
refinement  of  these  techniques  just  as  you  do,  in  many  cases,  in  your 
own  course  development  efforts. 
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SYSTEMS  APPROACHES  TO  THE  DESIGN  OF  CURRICULUM 


Mr.  Janes  L.  Foster 
U.  S.  Air  Defense  School 
Port  Bliss,  Texas  79916 


Good  morning.  I  would  like  to  introduce  my  presentation  on 
Systems  Approaches  to  the  Design  of  Curriculum  by  showing  you  a 
short  segment  of  the  Principles  and  Fractices  of  Instructional 
Technology  course.  This  is  a  15-lesson  sound-tape,  film  strip, 
and  programed  workbook  presentation  that  we  use  at  the  U.  S. 

Army  Air  Defense  School  as  part  of  an  orientation  to  systems 
engineering.  I  am  not  here  to  sell  the  course  although  it  has 
been  enthusiastically  received  at  Fort  Bliss  and  provides  an 
excellent  foundation  for  developing  systems  engineering  skills, 
particularly  in  the  area  of  constructing  behavioral  type  ob¬ 
jectives.  This  segment  I  a\»  about  to  show  provides  a  good  lead- 
in  to  the  design  of  instruction. 

How  then,  should  we  go  about  the  design  of  instruction  that 
will  result  in  effective  learning?  I  say  with  a  systems  approach. 
Why  a  systems  approach?  Well,  as  Dale  G.  Hamrens  says  in  the 
Oregon  State  System  of  Higher  Education  publication,  "The  Contri¬ 
bution  of  Behavioral  Science  to  Instructional  Technology,"  and  I 
quote,  "Because  it  is  the  most  powerful  and  efficient  means  pre¬ 
sently  available  for  determining  precise  learning  requirements  and 
arriving  at  the  most  effective  plan  for  eliciting  the  desired 
learning  outcomes  in  an  orderly  fashion." 

I  think  we  are  all  in  agreement  with  this  or  we  wouldn*t  be 
here.  A  systems  approach  provides  a  logical,  planned  development 
of  instruction.  It  facilitates  the  specification  of  training 
objectives  and  sub-objectives  in  terms  of  student  behavior. 

There  is  no  best  systems  approach.  A  multitude  of  literature 
on  systems  approaches  attests  to  this.  Let  us  briefly  examine 
some  of  these.  Some  of  you  head  a  status  report  on  systems 
engineering  as  it  is  presently  conducted  by  each  branch  of  the 
Armed  Forces  yesterday  afternoon  so  I  will  not  dwell  on  the  Army, 
Navy,  Marine  Corps  or  Air  Force  systems.  Let  me  outline  some 
other  approaches.  I  might  point  out  that  in  researching  for  this 
presentation,  I  found  that  all  systems  approaches  had  one  basic 
point  of  commonality;  that  terminal  objectives  and  sub-objectives 
must  first  be  established  and  the  instructional  system  designed 
from  there.  Or  as  H.  Del  Schalock  says  in  "The  Contribution  of 
Behavioral  Science  to  Instructional  Technology,"  "Instruction  takes 
its  focus,  content  and  often  its  form  from  the  nature  of  the  out¬ 
comes  that  is  being  pursued.  For  this  reason,  decisions  regarding 
the  design  of  instructional  experience  must  be  tied  to  learner 
outcomes." 

Dr.  Leslie  J.  Briggs,  in  his  "Handbook  of  Procedures  for  the 
design  of  Instruction,"  (1970)  has  developed  a  systems  model  for 
the  design  of  instruction.  He  points  out  that  there  is  a  charact¬ 
eristic  order  in  which  three  major  components  of  instruction  are 
designed. 
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(SHOW  SLIDE  #1) 


1.  Specification  of  instructional  objectives. 

2.  Development  of  tests  measuring  attainment  of  these 
objectives . 

3.  Selection  of  media  and  design  of  instructional 
materials . 

The  model  for  the  design  of  instruction  consists  of  10  steps: 

(SHOW  SLIDE  #2) 

1.  State  objectives  and  performance  standards. 

2.  Prepare  tests  over  the  objectives. 

3.  Analyze  objectives  for  structure  and  sequence. 

4.  Identify  assumed  entering  competencies. 

5.  Prepare  pre-tests  and  remedial  instruction. 

6.  Select  media  and  write  prescriptions. 

7.  Develop  first  draft  materials. 

8.  Small  group  tryouts  and  revisions. 

9.  Classroom  tryouts  and  revisions. 

10.  Performance  evaluation. 

Dr.  Briggs  emphasizes  the  importance  of  behavioral  type  objectives 
and  presents  their  development  as  a  three  step  process: 

1.  Writing  objectives  which  meet  three  criteria. 

2.  Selecting  appropriate  objectives. 

3.  Organizing  objectives  from  general  to  specific. 

The  first  step  points  up  the  necessity  of  objectives  meeting  the 
3  criteria  of  conditions,  task  and  standards.  In  other  words,  the 
objectives  must  convey  the  conditions  under  which  the  students  will 
perform;  the  specific  task  he  will  perform;  and  the  standards  of 
acceptable  performance. 

There  is  emphasis  or.  considering  the  learner's  needs  and  wished 
in  making  a  selection  of  appropriate  objectives.  It  also  points 
out  that  the  subject  areas,  research,  future  operations  and  experi¬ 
ence  should  also  contribute  to  making  selections.  Once  selected, 
the  objectives  must  be  organized  and  sequenced. 

(USE  BLOCKS  TO  SHOW  "FLAT"  STRUCTURE) 

This  might  take  the  form  of  a  "flat"  structure  with  unit  ob¬ 
jectives  resulting  in  course  objectives. 

(USE  BLOCKS  TO  SHOW  "HIERACHICAL"  STRUCTURE) 

Or  a  hierarchical  structure  where  subordinate  competencies  lead 
to  specific  behavioral  objectives  resulting  in  course  objectives. 

(USE  BLOCKS  TO  SHOW  "VERTICAL"  STRUCTURE) 

And  finally  the  sequencing  may  be  vertical  with  the  accomp¬ 
lishment  of  one  objective  leading  to  the  next  in  a  fixed  sequence. 

This  allows  for  the  transfer  of  learning  from  objective  to  objective. 

Once  the  course  objectives,  unit  objectives,  and  specific 
behavioral  objectives  have  been  developed,  evaluative  test  would  be 
prepared.  Dr.  Briggs  uggests  that  these  be  used  for  two  purposes: 

1.  For  tryouts  and  revisions  of  first  draft  materials,  thus 
evaluating  the  materials. 

2.  For  classroom  use  to  evaluate  student  performance  -  whether 
the  objectives  have  been  met.  This  will  also  contribute  to 
further  improvement  of  the  materials. 
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The  next  step  of  the  design  model  calls  for  analysis  of  the 
objectives*  so  that  instruction  may  be  appropriately  structured 
and  sequenced.  This  analysis  required  the  identification  of 
subordinate  competencies  down  to  the  lowest  level  or  until  each 
competency  represents  a  single  type  of  learning.  Then  it  calls 
for  identifying  the  types  of  learning  involved  in  the  competencies 
so  that  instruction  can  be  most  effectively  sequenced,  generally 
moving  from  the  simple  to  the  complex.  The  competencies  would 
be  numbered  in  the  order  in  which  they  will  be  taught.  It  is  then 
necessary  to  identify  the  entering  competencies,  those  things 
that  the  student  may  already  know  or  be  able  to  do.  Identification 
of  entering  competencies  would  require  renumbering  of  the  remain¬ 
ing  competencies. 

Dr.  Briggs  suggests  that  the  selection  of  media  should  give 
consideration  to  learner  characteristics.  This  would  be  quite 
difficult  in  military  instruction  where  students  have  such  varied 
backgrounds  and  experience.  Unless  we  were  considering  indivi¬ 
dualized  instruction.  In  this  model,  media  is  selected  for  each 
competency  of  each  objective.  After  media  selection,  a  prescrip¬ 
tion  is  developed  which  outlines  how  the  media  will  be  integrated 
with  the  subject  content  in  the  teaching  learning  situation. 

The  next  step  is  to  prepare  first  draft  materials  based  upon 
the  prescriptions  developed  in  the  previous  step.  The  final 
step  then  is  to  conduct  a  "formative”  evaluation  with  tryouts 
of  the  draft  materials  with  individuals  and  groups  of  learners, 
followed  by  tests  to  evaluate  the  materials  and  to  identify  needed 
revisions. 

This  particular  model  is  quite  similar  in  procedures  to  the 
systems  engineering  currently  practiced  by  CONARC  schools. 

Another  systems  approach  I  would  like  to  discuss  is  one  dev¬ 
eloped  by  Dr.  Sydney  J.  Drumheller  in  his  "Handbook  of  Curriculum 
Design  for  Individualized  Instruction  •  A  Systems  Approach.”  Like 
most  of  the  instructional  technologists.  Dr.  Drumheller  emphasized 
the  use  of  behavioral  objectives  which  will  result  in  student 
actions  which  are  both  obserable  and  evaluable.  He  divides  ob¬ 
jectives  into  two  categories,  "ends"  objectives  which  are  teriminal 
and  "means"  objectives  which  are  transitional. 

" E"  "M" 

(CITE  EXAMPLE  OP  SWIMMING  -  WATER WINGS) 

These  two  main  categories  are  further  divided  into  four  sub¬ 
categories: 

1.  Objective  complexes  -  one  or  more  terminal  objectives 
which  may  be  sub-divided  for  instructional  purposes. 

2.  Sub-objectives  -  used  when  larger  terminal  objectives 
are  broken  down  to  a  series  of  simpler  ones. 

3.  Prerequisite  objectives  -  relate  to  knowledges,  attitudes, 
or  skills  necessary  to  accomplish  the  terminal  objectives. 

4.  Isolated  objectives  -  "nice  to  knotf'  material. 
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Dr.  Drumheller  has  modified  Benjamin  Bloom's  Taxonomy  of  Ed¬ 
ucational  Objectives,  giving  it  a  stronger  structure  so  that  it 
can  be  used  to  identify  terminal  sub-objectives  when  the  terminal 
objectives  are  known.  He  modified  the  taxonomy  by  establishing 
six  major  categories  (of  objectives):  knowledge,  comprehension, 
application,  analysis,  synthesis  and  evaluation.  The  six  major 
categories  are  then  divided  into  sub-categories,  which  in  each 
case,  range  from  dealing  with  specifics  to  abstrations.  The 
taxonomy  can  then  be  used  to  analyze  behavior  and  identify 
objectives. 

In  designing  his  curriculum  model.  Dr.  Drumheller  established 
5  classifications  for  objectives: 

1.  Sequence  level  -  Defined  as  those  objectives  requiring  two 
or  more  courses  to  achieve. 

2.  Course  level  -  those  objectives  requiring  two  or  more  units 
to  achieve. 

3.  Unit  level  -  These  objectives  (from  the  Application  level 
of  the  taxonomy)  describe  the  integrated  complex  expected 
to  result  from  the  instruction. 

A.  Sub-unit  level  -  Sub-units  are  sequenced  into  a  procedure 
for  establishing  the  unit  terminal  behavior  complexes.  The 
sub-unit  objectives  will  focus  on  the  comprehension,  analysis, 
synthesis,  and  evaluation  levels  of  the  taxonomy.  The  final 
sub-unit  is  concerned  with  polishing  the  terminal  behaviors. 

5.  Rudimentary  level  -  composed  of  knowledge,  affective  and 

psychomotor  elements.  Rudiments  provide  the  focus  for  analysis, 
the  building  blocks  for  synthesis  and  a  sounding  board  for 
evaluation. 

There  can  be  anywhere  from  2-10  sub-units;  Dr.  Drumheller's 
model  consists  of  five.  Rudimentary  objectives  (knowledge)  are 
involved  in  all  sub-units.  The  first  sub-unit  serves  to  orient  the 
learner  to  the  unit  terminal  behavior  objectives  complexes.  He  will 
be  required  to  respond  using  knowledges  to  indicate  that  he  has  in 
fact,  identified  and  comprehends  these  terminal  objectives.  He  may 
recognize  some  sub-terminal  objectives  that  he  has  achieved  or  some 
that  he  still  needs.  He  is  not  expected  to  reach  any  sub-terminal 
objectives  in  sub-unit  II,  but  will  be  required  to  reach  a  group  of 
sub-terminal  objectives  in  each  of  the  other  sub-units. 

As  the  learner  progresses  through  the  various  sub-units,  the  be¬ 
haviors  from  previous  sub-units  are  included  to  provide  reinforcement. 
Thus,  as  the  learner  moves  to  higher  level  objectives,  there  is  auto¬ 
matic  reinforcement  of  lower  level  behaviors. 

The  second,  third,  and  fourth  sub-units  center  on  analysis, 
synthesis,  and  evaluation  respectively.  In  analysis,  the  learner 
identifies  elements,  relationships,  and  organizational  principles 
in  simulated  tasks  related  to  the  complexes.  In  synthesis  he  sy¬ 
nthesizes  complex-oriented  responses  to  simulated  or  contrived  tasks 
related  to  the  complexes.  And  in  evaluation  he  appraises  performances 
by  himself  or  otners,  on  tasks  related  to  the  terminal  behavior 
complexes. 
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The  fifth  or  last  sub-unit  is  devoted  to  polishing  the  learner*s 
performance  until  he  demonstrates  the  terminal  behavior.  It  is  the 
only  sub-unit  that  has  mastery  of  the  behavior  complex  as  a  major 
emphasis.  When  a  student  can  perform  at  this  level  of  proficiency 
he  has  completed  the  unit. 

Dr.  Drumheller  points  our  that  some  "nice  to  knotf*  objectives 
may  be  desirable  to  interest  and  motivate  the  student;  but  he  em¬ 
phasizes  that  too  many  "nice  to  knot/'  (isolated)  objectives  will 
degrade  the  unit  of  instruction. 

There  are  numerous  design  models  that  could  be  discussed  and 
analyzed,  but  those  1  have  examined  are  quite  similar  in  procedures, 
and  the  two  I  have  outlined  here  are  typical.  The  question  is  -  will 
current  system  approaches  produce  the  courses  of  instruction  we 
need?  There  is  no  doubt  that  all  of  the  approaches,  properly  em¬ 
ployed  would  result  in  well  constructed  courses  of  instruction; 
however,  I  believe  that  military  instruction  needs  a  stronger  orientation 
toward  an  individualized  approach.  Dr.  Drumheller’s  design  model 
does  this  to  a  large  extent  but  I  think  there  must  be  greater  emphasis 
on  selfpacing. 

To  accomplish  this,  media  selection  takes  on  a  prime  importance 
and  I  think  it  should  come  earlier  in  the  design  approach  than  it 
does  in  most  models.  I  feel  that  media  selection  is  the  key  to 
affective  self-paced  individualized  instruction.  Perhaps  we  should 
be  giving  much  more  consideration  to  what  Robert  Heinich  calls 
"mediated”  instruction  in  his  monograph.  "The  Systems  Engineering 
of  Education  II:  Application  of  Systems  Thinking  to  Instruction." 

He  distinguishes  between  "mediated"  instruction  (where  the  media 
teacher  presents  the  instruction).  He  points  out  that  too  often 
in  the  past,  media  were  considered  only  as  "aida"  to  the  instruc¬ 
tion,  which  may  be  due  to  "the  tradition  of  the  treacher  as  the  sole 
arbiter  of  what  is  used  in  the  classroom."  As  instructional  tech¬ 
nology  and  media  development  have  advanced,  the  teacher  has  been 
fearful  that  he  would  lose  his  classroom  authority  and  responsibility 
to  the  media.  I  am  sure  that  all  of  you  can  recall  much  instructor 
grumbling  when  military  schools  adopted  educational  television. 

Dr.  Heinich  recommends  a  shared  responsibility  between  the  teacher  and 
the  media  and  as  he  states,  "the  teacher  of  the  future  will  be  a 
very  vital  subsystem  in  the  total  instructional  system  and  he  must 
be  trained  to  function  in  that  capacity." 

Dr.  Heinich  feels  that  technology  will  force  the  transfer  of 
classroom  teachers  from  one  side  of  "mediation"  to  the  other,  and 
he  predicts  that  in  20  years  (1985)perhaps  1/3  of  the  teaching 
profession  will  be  engaged  in  preparing  materials  with  little  or 
no  direct  face  to  face  contact  with  students. 

Since  1965  when  Dr.  Heinich* s  monograph  was  published,  events 
would  certainly  seem  to  support  his  prediction.  Great  strides  are 
being  made  in  computer  assisted  instruction  and  the  use  of  educational 
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television  and  programed  textbooks  has  been  expanded.  I  be live 
that  this  is  the  direction  our  systems  design  must  take  in  the 
imnediate  future,  more  emphasis  on  individualized  instruction 
with  greater  use  of  media  which  will  permit  self-pacing. 

Certainly  a  systems  approach  is  the  most  efficient  way  to 
design  curricula,  but  with  students  of  such  varied  background, 
experience,  and  ability  such  as  we  have  in  military  schools 
the  instruction  must  be  geared  to  the  individual,  and  self- 
paced,  individualized  instruction  appears  to  be  the  answer. 

At  any  rate  there  is  a  fertile  field  for  educational  tech¬ 
nologists  and  instructional  designers  to  experiment  in  the 
development  of  a  design  model  oriented  toward  recognition  of 
individual  differences  and  the  use  of  a  variety  of  media 
which  will  permit  maximal  self-pacing  and  produce  a  well 
trained,  effective  product. 
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ADMINISTRATIVE  INNOVATIONS 


Dr.  Philip  M.  Tiemann 
U.  S.  Amy  Medical  Department 
Veterinary  School 
Chicago,  Illinois  60609 


c 


The  Medical  Department  Veterinary  School  initiated  a  long  range 
plan  for  further  development  of  its  instruction  in  the  Fall  of  1967. 
The  plan  was  based  upon  a  systematic  model  of  the  school's  mission. 
Briefly,  we  identified  two  areas  of  concern  —  the  school,  at  left, 
and  the  real  world,  at  the  right  —  with  trainees  turned  out  as  job 
incumbents.  The  key  udefulness  of  our  model  is  that  we  acknow- 
ledged  making  a  host  of  assumptions.  We  assumed  that  exposing  trains 
ees  to  instruction  prepared  them  to  contribute  on-the-job .  Out  long 
range  plan  was  simply  to  develop  the  mechanisms  to  get  some  formal 
feedback  from  the  real  world  to  verify  assumptions  we  were  making. 

A  first  step  was  to  identify  the  level  of  performance  being  ac¬ 
hieved  by  our  present  trainees.  Feedback  from  the  real  world  cannot 
verify  anything  if  we  don't  know  where  we  already  are.  Notice  some¬ 
thing  else.  Our  assumptions  have  been  based  upon  the  best  advice 
available  officer  and  enliated  instructors  with  years  of  experience 
conscientiously  design  and  offer  the  best  instruction  they  can. 

There's  a  lot  of  good  going  on  here.  But  it  doesn't  hurt  anyone  to 
verify  that? 

We  had  just  begun  to  implement  these  ideas  when  we  came  upon 
CONARC  Reg  350-100-1.  It  provided  support  for  the  plan  —  even 
a  term  for  our  verifying  assumptions.  Quality  Control.  It  also 
posed  some  problems.  We're  a  small  school.  Trying  to  implement 
systems  engineering  —  SE  — .  is  like  the  crew  of  a  speeding  train 
trying  to  remodel  while  enroute.  Nothing  can  Stop  while  the  steps 
of  SE  proceed. 

Our  first  administrative  problem  was  and  still  is  to  generate 
what  the  academic  community  calls  ’’release  time"  -—  to  get  some 
of  our  people  away  from  the  pressing  routine  —  free  to  analyze 
tasks,  to  upgrade  performance  tests,  and  to  improve  instruction  — 
hopefully  by  putting  together  what  I  call  instructional  capital  — 
carefully  developed  materials  or  sequences  of  known  effectiveness 
which  can  be  re-used,  preferably  by  other  instructors.  In  short, 
such  material  or  sequences  support  instructors  in  attaining  success  — 
their  trainees  succeed. 

Another  administrative  problem  is  that  the  steps  of  SE  must  be 
completed  by  people  who  have  some  idea  of  what  they're  about  —  or 
all  the  carefully  generated  release  time  is  wasted.  So  we  began 
in-house  training  on  SE  —  learning  how  to  do  it  while  slowly 
applying  the  steps  to  our  basic  enlisted  course.  Ve  began  with  the 
basic  course  —  reasoning  that  any  increase  in  trainee  skills  would 
influence  the  SE  effort  in  all  other  courses,  that  is,  the  training 
of  supervisors  depends  upon  how  well  their  men  are  able  to  perform. 

During  the  planning  phases  of  SE,  we  ran  into  one  huge  problem. 
Our  existing  organization  was  based  upon  major  subjects  we  taught 
red  meats,  fruits  and  vegetables,  and  so  on.  We  found  little  em¬ 
pires  and  these  had  us  working  at  cross  purposes  —  with  effort 
wasted  in  duplication  and  overlap. 


*  A  presentation  by  the  U.S»  Army  Medical  Department  Veterinary 
School  to  the  OONARC  Systems  Engineering  of  Training  Workshop,  Ft. 
Gordon,  Georgia,  October  5-7,  1971. 
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Basically,  we  found  responsibility  for  instructional  policy 
decisions  at  the  command  level  and  we  found  these  decisions  being 
made  at  the  bottom  level  —  by  the  instructors,  and  in  some  cases, 
other  support  personnel.  How  did  this  happen?  Well,  our  POLs 
contained  the  usual  general  remarks  providing  no  specific  guid- 
ance.  By  design,  instructors  did  their  own  thing.  They  wrote 
lesson  plans,  conducted  instruction,  and  evaluated  trainee  per- 
formance  with  their  own  tests.  Class  monitoring  revealed  charm, 
platform  manner ,  the  level  of  ventilation  and  other  useless  in¬ 
formation  —  but  command  was  not  sure  what  the  instruction  ac¬ 
complished  in  that  closed  loop  at  the  bottom. 

We  dealt  with  these  problems  in  two  way 3.  We  created  an  Ad 
Hoc  Committee  for  SE,  which  1*11  return  to  in  a  moment,  and  we 
recognized  the  need  for  a  structure  to  support  SE  —  so  we  designed 
one.  In  place  of  the  almost  autonomous  subject-oriented  depart¬ 
ments,  we  planned  two  instructional  and  three  direct  support  de¬ 
partments.  We  implemented  this  in  January  of  1970.  After  the 
initial  shock,  things  began  to  work  and,  strangely  enough,  to 
work  better  —  to  eliminate  much  of  the  duplication  and  overlap. 

As  we  regained  our  nerve,  we  began  to  talk  about  a  SE  department. 

Now,  there*s  nothing  in  the  staffing  guide  which  specifi¬ 
cally  provides  for  such  a  creature  —  so  we  pulled  together  all 
related  duties  and,  in  our  manpower  survey  in  August  of  *71, 
successfully  justified  our  SE  department.  A  duty  description  is 
included  in  your  handout. 

Let  me  turn  to  the  second  issue.  The  command  created  an  Ad 
Hoc  Committee  for  Se  consisting  of  the  DI  and  the  five  depart¬ 
ment  heads.  The  Committee  conducts  praliminary  review  of  task 
analysis  data  and  complete  review  of  any  revision  in  training 
objectives  and  testing.  After  review  and  command  approval,  in¬ 
struction  redesign  may  begin. 

Our  task  analysis  ars  taught  to  apply  behavior  analysis  pro¬ 
cedures  that  is,  to  identify  the  types  of  learning  in  order  to 
specify  appropriate  conditions  of  instruction  and  testing  of 
trainees.  When  we  train  our  analysts,  we  sort  out  types  of  learn¬ 
ing  following  a  modification  of  Professor  Gagne*  taxonomy.  Look¬ 
ing  at  stimulus-response  pairs  as  basic  units,  our  analysts 
record  and  report  their  observations  in  terms  of  such  discrimin¬ 
ations  which  trainees  must  make.  In  the  case  of  multiple  dis¬ 
criminations,  S-R  pairs  may  be  represented  in  a  table  —  or,  when 
sequential  tasks  are  involved,  as  chains  or  serial  memory. 

Beyond  these  types,  the  analysts  deal  witn  complex  cognitive 
learning  —  types  which  require  trainees  to  deal  with  new  sit¬ 
uations  not  met  in  training.  In  other  words,  these  are  always 
beyond  memory  —  no  matter  how  efficient  memory  might  be.  Some 
of  you  may  refer  to  this  as  the  ability  of  training  to  transfer 
to  new  situations.  Above  concept  learning,  we  identify  principle 
applying  —  equal  to  Gagne’s  ruls  lsarning  —  and  problem-solving, 
situations  which  may  require  trainees  to  find  —  to  discover  — 
the  rules  which  apply. 
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Once  tasks  have  been  described  in  such  terms,  training  policy 
decisions  can  be  made  in  a  rational  way.  But  notice  the  problem 
here.  The  language  of  behavior  analysis  is  foreign  to  those  who 
must  make  decisions.  For  this  reason,  the  Ad  Hoc  Committee  mem¬ 
bers  have  spent  some  50  hours  in  SB  instruction.  In  effect,  they 
learn  to  translate  analysis  information  into  policy. 

Here's  an  example  —  a  summary  of  an  actual  Committee  review 
of  a  job  of  all  enlisted  veterinary  inspectors  —  that  of  prepara¬ 
tion,  use,  and  safeguard  of  their  government  inspection  stamp. 

In  your  handout,  portions  of  the  analysis  are  reproduced. 

The  first  type  —  content  analysis  —  was  based  upon  existing 
documents  and  instruction.  It  identified  several  types  of  learn¬ 
ing.  As  example,  we  include  the  behavior  chain  of  setting  up  an 
inspection  stamp  with  required  numbers. 

An  example  of  a  multiple  discrimination  is  recognition  of  sev¬ 
eral  inspection  stamp.  An  example  of  a  conceptual  task  is  under¬ 
standing  what  "sufficient"  means  with  respect  to  the  rule  that 
inspectors  must  stamp  a  sufFicient  number  of  items  in  an  inspected 
1  it. 

To  translate  such  information  into  policy,  the  Committee's  first 
step  is  to  require  verification  that  a  training  need  exists.  Work¬ 
ing  from  the  content  analysis,  our  analysts  contacted  supervisory 
personnel  in  the  field  to  verify  a  training  need.  The  results  of 
interview  analysis  are  summarized  in  your  handout.  Briefly,  super¬ 
visors  want  trainees  to  know  when  to  stamp  what,  and  how  much,  and 
how  to  safeguard  their  stamp. 

The  Ad  Hoc  Committee's  next  step  is  to  look  at  the  fit  between 
content  and  interview  analysis  data.  To  their  dismay,  they  noted 
in  the  first  case  that  the  most  significant  performance  expected  of 
a  trainee  —  that  he  know  when  to  stamp  —  was  completely  assumed 
and  never  taught  or  tested  during  formal  instruction.  An  objective 
was  written  and  the  evaluation  department  directed  to  prepare  per¬ 
formance  tests  for  the  training  objective. 

In  the  second  case,  interview  analysis  did  verify  the  "how  much" 
requirement  of  stamping.  But  the  committee  found  that  existing 
instruction  merely/  caused  the  trainee  to  memorize  the  official 
directive  —  the  requirement  that  a  "sufficient"  number  of  items 
be  stamped.  The  problem  is  —  what  is  the  trainee's  concept  of 
"sufficient"? 

Here's  a  slide  from  our  Jabberwock  program  teaching  the  how- 
to-do-it  aspect  of  dealing  with  concept  learning.  A  trainee  on- 
the-job-  might  not  recognize  "sufficient"  if  it  came  up  and  bit 
him. 

The  Ad  Hoc  Committee  is  able  to  locate  inadequacies  of  this 
type  and  take  corrective  action  when  its  members  are  able  to 
recognize  the  different  types  of  learning  involved.  Corrective 
action  required  visual  examples  of  "sufficient"  —  a  variety  — 
ala. i  examples  of  "insufficient"  —  to  make  sure  trainees  have  the 
conceit. 
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As  a  third  example,  the  Committee  found  that  existing  instruction 
in  recognizing  a  variety  of  stamps  was  not  verified  by  interview 
analysis,  but  directives  do  impose  a  job  requirement  in  this  respect. 
The  Committee  recommended  that  existing  training  on  this  task  be  re¬ 
placed  by  a  job  aid  —  a  wallet-sized  card  with  these  stamps  and 
their  names  —  issued  to  each  man. 

In  the  fourth  case,  locating  training  requirements  not  disclosed 
by  any  analysis  is  a  problem  irrespective  of  the  system  of  training 
design  we  use.  But  given  the  analysis  thus  far,  the  Committee  did 
notice  a  few  oversights  —  for  instance,  teaching  a  trainee  the 
proper  way  to  ink  a  pad,  a  behavior  chain. 

Analysis  procedures  which  identify  job  requirements  by  type  of 
learning  can  be  used  to  verify  training  requirements,  to  identify 
areas  of  under  and  over  training,  and  to  make  rational  training  policy 
decisions . 

Notice  how  our  organization  keeps  these  decisions  from  slipping 
into  the  old  closed-loop  at  the  bottom.  Training  objectives  go  to 
evaluation  as  well  as  to  instruction  departments.  Evaluation  both 
prepares  and  administers  our  testing.  If  trainee  performance  is 
inadequate,  the  whole  process  is  open  to  command  review.  Are  the 
tests  appropriate  to  the  objectives?  If  so,  what*s  the  problem  with 
instruction?  Is  time  inadequate?  Are  procedures  at  fault? 

Enlightened  command  reaction  must  prevail.  Putting  the  finger 
on  instructors  is  not  our  purpose.  It  is  to  provide  administrative 
support  to  release  instructors  to  work  with  SE  people  to  improve  the 
quality  of  instruction.  Also  at  this  point,  diagnosis  of  individual 
trainee  testing  has  resulted  in  a  weekly  retraining  procedure 
followed  by  retesting.  This  is  one  of  the  innovations  whitoh  has 
helped  to  cut  our  academic  loss  of  trainees  by  fifty  percent. 

Returning  to  our  new  structure  --  we  were  ready  to  SE  with 
effect.  The  next  question  was,  "Who  does  the  work?"  Here’s  the 
course  schedule  we  faced  in  *69.  Eleven  basic  enlisted  courses. 

Three  three-week  refresher  courses.  Three-  ten-week  advanced  en¬ 
listed  courses.  Five  basic  officer  courses  and  a  variety  of  ad¬ 
vanced  and  special  officer  courses. 

Look  at  just  the  enlisted  situation.  Usually,  there  are  four 
courses  at  once  and  the  12  instructors  who  cover  these  average 
over  13  hours  on-the-stump  each  week.  That  jumps  to  about  17  hours 
in  June.  And  each  man  teaches  his  specialty  in  three  courses.  The 
changes  in  scheduling  overlap  —  the  shift  of  relative  convening 
weeks  —  causes  constant  shuffling  of  course  sequencing  so  some¬ 
one  is  not  teaching  in  two  or  three  courses  at  once.  Similar  pro¬ 
blems  exist  with  respect  to  lab  space  scheduling. 

In  addition,  SE  is  more  than  filling  out  task  analysis  forms, 
rewriting  objectives,  lesson  plans,  and  POIs.  The  guts  of  SE  is 
providing  instructors  with  adequate  support  so  they  can  stop  lect¬ 
uring  in  lab  periods.  It*s  getting  trainee  hands  on  equipment  -- 
their  butts  out  of  the  chairs.  It*s  realistic  performance  testing. 
This  means  changing  the  mix  of  instructor  scheduling  —  small  grouping 
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for  laos,  some  big  lectures,  and  some  multiple  instructors  assigned 
as  resource  people  during  practicals.  It  means  consistent  advance 
planning  to  schedule  the  precious  resource  of  instructor  time  in  a 
manner  dictated  by  the  type  of  learning  involved  during  each  hour 
of  each  day  in  each  course.  The  guts  of  SE  requires  that  instructor 
scheduling  must  float  free  from  the  outmoded  concept  of  the  platform 
hour  —  and  this  is  more  traumatic  than  the  dollar  floating  free 
from  gold. 

But  no  flexibility  when  assigning  instructors  or  rational  seq¬ 
uencing  of  instruction  can  prevail  with  this  kind  of  course  sch¬ 
eduling.  But  the  resistance  to  consider  change  was  unbelievable. 

For  discussion,  we  put  together  the  optimum  schedule.  We  put  the 
basic  and  advanced  courses  in  a  set  pattern  to  get  instructor 
scheduling  under  control,  planned  this  ten-week  sequence  five  times 
a  year,  and  planned  to  double  the  input  into  the  basic  course  — 
required  by  reducing  from  ten  to  five  the  number  of  times  it  was 
offered. 

I  won*t  go  into  the  traumatic  days  of  selling  this  idea.  Most 
valid  objections  turned  out  to  be  instances  of  the  administrative 
tail  wagging  the  training  dog  —  for  example,  insistence  we  couldn*t 
work  out  pay  records  in  the  necessary  week.  We  developed  a  procedure 
spreading  that  job  out  over  the  entire  eight  weeks  —  and  pay  people 
are  delighted. 

The  most  frequent  question  asked  is  why  Washington  bought  this 
plan.  Basically,  because  they  have  more  flexibility  now.  No  longer 
do  they  fuss  with  adding  or  cancelling  scheduled  classes.  They 
simply  increase  or  decrease  input  into  the  basic  classes  —  knowing 
we  are  organized  to  accept  up  to  110  students  with  our  concept  of 
variable  sections  within  courses. 

It  took  some  time  to  get  the  new  schedule  into  operation  «—  with 
the  advanced  courses  back  to  back  through  the  year.  Only  four  will 
be  offered  in  *70.  We  began  to  look  at  our  advanced  courses  in 
relationships  to  our  three  different  refresher  courses.  For  a  variety 
of  good  reasons,  we  divided  the  advanced  course  into  five  two-week 
modules.  Then  we  expanded  the  total  refresher  course  from  three 
separate  three  week  courses  —  nine  weeks,  to  ten  weeks,  divided 
into  the  same  modules. 

Now  we  offer  field  commanders  a  flexible  option.  They  can  send 
a  man  for  refresher  training  by  topic  --  for  any  number  of  these 
five  modules.  As  time  permits,  the  man  can  return  for  remaining 
modules. 

With  respect  to  rational  systems  engineering,  our  options  are 
now  open.  We  schedule  our  twelve  enlisted  instructors  to  accomo¬ 
date  the  learning  requirements  of  these  two  courses.  They  always 
occue  in  the  same  configuration  so  weaknesses  in  scheduling  may 
be  corrected  the  following  period. 

We  are  releasing  staff  time  for  SE  —  witn  a  department  to  work 
closely  with  instructors,  hopefully  for  developing  the  kind  of  high 
quality,  resuable  instructional  capital  that  can  be  upgraded  cont¬ 
inuously.  Individually,  these  are  little  things  --  sequences  of 
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color  slides  to  support  visual  discrimination  training  in  the  labs, 
handout  materials  of  proven  effectiveness,  short  self- instructional 
lessons  —  but  the  cumulative  effect  of  such  bit  by  bit  effort  will 
improve  the  whole  school.  The  intent  of  systems  engineering  of 
training  is  instructional  innovation  —  and  that  seems  to  be  about 
90  per  cent  administrative  innovation. 
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SOFT  SKILL  COURSES 


Mr.  Charles  Harvey 

U.  S.  Army  Infantry  School 

Fort  Benming,  Georgia  31905 


THE  USAIS  APPROACH  TO  SYSTEMS  ENGINEERING  SOFT  SKILL  COURSES 
Topics  discussed  will  be  as  follows: 

1.  How  do  you  list  tactical  concepts  and  leadership  qualities  or 
characteristics  in  a  task  inventory? 

We  list  them  as  supporting  skills,  knowledges,  or  attitudes  which 
are  later  applied  during  CPX'S,  FTX*S  and  other  training. 

2.  How  do  you  establish  the  job  standard  for  such  tasks  as  take 
an  enemy  position,  prevent  awol*s  and  stop  the  bleeding? 

We  have  experienced  combat  veterans  advise  us  in  establishing 
implied  or  derived  standards. 

3.  How  do  you  establish  valid  training  conditions  and  training 
standards? 

We  try  to  simulate  realistic  situations  as  much  as  possible. 

We  must  establish  attainable  and  maintainable  standards,  which 
are  usually  short  of  perfection, 

4.  How  do  you  evaluate  the  student*s  achievement  of  Soft-^kill 
objectives  and  the  effectiveness  of  the  course  in  teaching  these 
objectives? 

Although  some  objective  tests  are  given,  evaluation  is  mostly 
subjective.  Corrective  actions  are  taken  immediately  when  course 
structures  are  defective. 

This  discussion  is  followed  by  a  brief  review  of  the  three  forms 
USAIS  developed  for  use  in  systems  engineering  its  courses: 

1.  FB  Form  125  -  Task/Subtask  (SKA)  inventory  and  selection  sheet 
This  form  serves  a  dual  purpose  in  listing  both  task  inventories 

and  SKA  inventories. 

2.  FB  Form  126  -  Training  analysis  information  sheet 

This  form  serves  to  record  the  training  objective,  the  criterion 
and  the  learning  analysis. 

3.  FB  Form  24  -  Systems  engineering  course  design  sheet 

This  form  summarizes  the  training  objectives  into  lessons  and 
describes  the  manner  in  which  the  class  is  conducted. 


COMPUTERIZED  ASPECTS  OF  SYSTEMS  ENGINEERING 


Dr.  Harold  Wagner 
Human  Resources  Research 
Organization  (HumRRO) 

300  North  Washington  Street 
Alexandria,  Virginia  22314 


Mr.  James  E.  Henry 

U.  S.  Quartermaster  School 

Fort  Lee,  Virginia  23801 


DEFINITION  FOR  A  TASK  I NV 04 TORY 

You  are  already  quite  familiar  with  the  concept  of  the  task 
inventory  and  the  fact  each  task  has  associated  with  it  certain 
skills,  knowledges,  and  attitudes  that  must  be  present  if  the 
task  is  to  be  carried  out  successfully.  Each  school,  l*m  sure, 
has  developed  a  technique  that  works  reasonably  well  for  it  in 
managing  the  rather  voluminous  data  that*s  accumulated  in  this 
aspect  of  systems  engineering. 

We  would  like  to  share  with  you  the  concept  of  the  automated 
task  inventory  that  was  developed  jointly  by  the  Quartermaster 
School  and  Hum  RRO  in  Project  STOCK.  We  have  applied  the  tech¬ 
nique  to  two  MOS*s  for  which  we  are  proponent.  There  is  still 
work  to  be  done  on  one  of  the  courses,  but  the  automated  task 
inventory  technique  is  operational  at  HumRRO. 

When  systems  engineering  first  began  there  was  a  considerable 
debate  over  what  constituted  a  task— how  grossly  or  how  defini¬ 
tively  an  action  should  be  described  in  a  task  inventory.  We 
tried  to  position  ourselves  somewhere  between  statements  such  as 
the  following: 

Large;  Operate  a  storage  facility 
Small:  Enter  quantity  on  card 

Some  of  you  may  have  arrived  at  definitions  that  you  can  live 
with,  but  as  far  as  1  know  a  good  definition  of  a  task  has  been 
a  bit  elusive.  My  feeling,  after  admittedly  inadequate  sampling, 
is  that  task  statements  have  tended  to  be  fairly  large  and  broad 
in  other  schools.  As  you  will  see,  we  have  adopted  the  best  of  both 
extremes.  Now  the  concept  we  have  used  is  a  fairly  simple  one  and, 
while  it  doesn*t  define  a  task,  it  does  help  to  define  what  a 
task  inventory  ought  to  look  like. 

The  task  inventory  is  not  just  a  rec ord— a  list— of  what  is 
being  done  by  the  man  who  holds  an  MOS. 

It  is  not  just  a  source  of  information  for  a  questionnaire 
that  will  go  to  the  field. 

Most  importantly,  the  task  inventory  is  a  means  of  communication 
initially  between  the  systems  engineer  or  analyst  and  the  guy  who  is 
responsible  for  developing  a  course:  the  writer  of  the  lesson  plan, 
the  instructor. 

Ultimately  the  task  inventory  is  a  means  of  communication  between 
the  course  director  and  the  instructors  and  other  members  of  his 
3taff  who  design  the  methods,  teach  tKe  course  a'nc^  evaluate  the 
students  on  a  day  to  day  basis.  What  goes  into  that  task  inventory 
must  make  eminently  good  sense  in  terms  of  the  kind  of  communication 
required  between  the  course  director  and  the  instructional  staff. 

HOW  TO  STRUCTURE  A  TA&K  II^VENTORY 

I  have  seen  task  lists  that  looked  like  grocery  shopping  lists. 
Each  task  appeared  to  be  listed  randomly  as  it  occured  to  the 
analyst. 

As  a  result  each  task  tended  to  stand  alone  and  independent  of 
those  that  preceded  and  followed,  and  each  task  appeared  to  be  of 
equal  value  as  far  as  its  importance  was  conoemci.  There  is  no 
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built  in  value  discrimination  in  grocery  list  outlines.  For  that 
reason  systems  engineering  includes  a  step  called  "Selection  of 
Tasks  for  Training",  and  under  "Training  Analysis"  we  worry  about 
"clustering"  and  "sequencing."  Now  we  can*t  eliminate  those  steps 
completely  but  we  can  simplify  them  by  organization  of  the  task 
inventory . 

Instead  of  picturing  a  grocery  shopping  list  in  which  each 
item  on  the  list  is  more  or  less  independent  of  the  other  items, 
imagine  an  outline  format  following  somewhat  the  same  principles 
of  outlining  that  you  learned  in  freshman  high  school  English. 
(Remember,  our  intent  is  to  communicate  to  the  instructional  staff 
what  it  is  that  needs  to  be  taught  and  what  skills  and  knowledge 
need  to  be  present  in  order  for  a  student  to  perform  the  task 
required  of  him.) 

We  might  visualize  a  task  for  one  of  our  supply  MOS's,  for 
example,  that  would  read  something  like  this: 

SLIDE  5 

"Issues  non-expendable  supplies."  Now  this  task  might  satisfy 
most  task  definitions  but  it  falls  short  of  conmunicating  all  we 
would  like  to  know  about  what  must  be  taught  and  how  it  must  be 
taught  and  evaluated.  That  task  be  itself  ignores  the  many  steps 
that  must  be  taken  to  perform  it  and  the  skills  and  knowledges 
that  support  it. 

SLIDE  6 

It  ignores  the  reference  documents  that  must  be  understood  and 
handled  to  perform  the  task. 

SLIDE  7 

It  ignores  the  paper  work  that  must  be  managed  to  perform  the 
task. 

SLIDE  8 

It  tells  us  nothing  about  the  skills  required  in  locating 
alphabetic  and  numeric  information,  or 
SLIDE  9 

in  being  able  to  operate  a  typewriter  at  a  minimal  level. 

SLIDE  10 

It  tells  us  nothing  about  the  student's  ability  to  transcribe 
information  accurately  or 
SLIDE  11 

to  do  simple  arithmetic. 

It  does  not  tell  us  what  part  of  the  total  operation  is  critical 
enough  to  demand  virtually  perfect  performance  (such  as  transcribing 
Federal  Stock  Numbers),  and  what  part  of  the  operation  can  permit 
error  (such  as  the  spelling  of  nomenclature). 

SLIDE  12 

Our  hierarchical  task  list,  then,  structured  in  the  manner  of 
a  freshman  English  outline,  takes  the  major  task  and  subdivides  it 
into  as  many  levels  of  subtasks  as  are  required  to  define  the  task 
exactly.  Broadly  stated 
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SLIDE  13 

tasks  are  followed  by  successively  smaller  subtaks  that  des¬ 
cribe  the  performance  of  the  broader  task  in  enough  detail  that 
the  instructor  will  know  what  must  be  taught  and  tested. 

SLIDE  14  (BLANK) 

For  that  reason  it  serves  as  a  rather  exact  meand  of  communi¬ 
cation  between  a  course  director  and  his  instructional  staff.  But 
because  of  the  outline  structure,  the  course  director  need  only 
deal  with  the  levels  of  detail  (perhaps  the  broadly  stated  tasks) 
that  suit  his  purpose.  The  numbering  system,  as  you  will  see,  makes 
this  possible.  The  fact  that  the  task  inventory  is  quite  definitive 
does  not  prohibit  variations  in  the  sequence  of  objectives  when 
variations  make  good  sense  from  an  instructional  point  of  view. 

TRAINING  IN  FUNCTIONAL  CONTEXT 

”  Now  we  have  adopted  the  concept  of  "functional  context"  training, 
SLIDE  15 

which  means  that  we  feel  that  a  student  who  is  learning  skills 
for  particular  circumstances  will,  in  general,  learn  better  and  faster 
if  the  circumstances  are  present  in  the  learning  situation. 

SLIDE  16 

When  a  job  is  analyzed  into  its  subject  matter  content  (rather 
than  into  genuine  tasks)  the  student  finas  himself  in  a  conventional 
classroom  learning  principles  and  theories  tthat  he  may  not  need  when 
he  goes  to  work.  (I  should  point  out  that  the  act  of  making  a  task 
list  doesn*t  guarantee  that  the  tasks  will  be  job  related.)  "Functional 
context"  also  means  that  when  particular  skills  and  knowledge  must 
be  taught  the  student,  they  should  be  taught  at  the  time  they  are  needed 
and  to  the  extent  needed  for  the  student  to  be  able  to  perform  the  tasks 
successfully. 

SLIDE  17 

When  it*s  necessary  for  a  student  to  receive  specific  instruction 
in  connection  with  his  task  training,  he  should  get  the  individual 
treatment  he  requires.  On-the-job  training  is  "functional  context" 
training.  We  would  like  for  our  classroom  training,  in  time,  to 
simulate  real  life  in  as  realistic  a  fashion  as  possible.  "Functional 
context"  training  brings  on-the-job  training  to  the  classroom. 

SLIDE  18 

Our  task  list  then  must  analyze  taska  as  they  are  performed  in 
enough  detail  to  communicate  clearly.  The  task  list  must  be  care- 
fully  supplemented  with  the 
SLIDE  19 

skills  and  knowledges  required  to  support  each  task,  and  the  two 
together— tasks  together  with  skills  and  knowledges— 

SLIDE  20  (BLANC) 

define  for  the  instructor  the  content  of  training. 

COURSE  MANAGEMENT  WITH  A  COMPUTER 

Now  we*ve  indicated  that  our  task  inventory  would  be  computerized 
and  handouts  have  been  provided  showing  a  typical  computer  listing  of 
items  in  the  task  inventory. 
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The  fact  that  the  task  inventory  has  been  computerized  in 
no  way  eliminates  the  basic  effort  required  in  creating  the  task 
inventory  to  begin  with.  It  seems  apparent  to  us  that  course 
management  after  the  task  inventory  has  been  completed  will  proceed 
in  a  much  more  orderly  fashion  and  that  work  will  eventually  be 
saved.  This  will  become  apparent  when  Dr.  Wagner  shows  you  just 
what  the  task  inventory  includes.  The  tasks  and  subtasks  listed 
in  outline  fashion  represent  a  kind  of  logic  that  permits  the 
addition  or  deletion  of  entire  blocks  if  necessary.  The  course 
director  will  be  able  to  study  course  logic  and  alter  it  as  he 
sees  fit.  The  problem  of  updating  a  systems  engineered  course 
has  lacked  a  completely  satisfactory  solution  until  now.  With  this 
printout  the  course  director  will  have  a  highly  visible  outline 
of  the  course  in  front  of  him  that  will  let  him  update  any  or  all  of  his 
systems-engineered  course  at  any  time. 

Having  a  course  structured  in  the  kind  of  outline  form  I  have 
described  serves  two  other  important  purposes. 

In  the  first  place  the  "functional  context”  structure  describes 
in  considerable  detail  the  way  in  which  practical  exercises,  problems, 
or  simulations  should  be  developed  in  order  to  be  thoroughly  realistic. 

The  second  benefit  to  be  gained  from  the  outline  structure  is  that 
tests,  examinations,  and  graded  practical  exercises  should  be  per¬ 
formance  oriented  following  the  logic  of  the  outline.  Evaluation 
instruments,  if  they  follow  this  kind  of  logic,  should  tell  us 
clearly  whether  or  not  the  student  is  qualified  in  the  job  we  are 
trying  to  teach  him  to  do. 

Now  I  would  like  to  ask  Dr.  Wagner  from  HumRRO  to  tell  you  about 
the  entries  that  are  included  on  our  automated  task  inventory. 
DESCRIPTION  OF  TECHNIQUE 

Up  to  this  point,  Mr.  Henry  has  discussed  our  approach  to  the  dev¬ 
elopment  of  an  automated  task  inventory,  and  the  structure  that  we  have 
imposed  upon  it  by  employing  the  concept  of  functional  contept  of  funct¬ 
ional  context  training. 

I  would  now  like  to  discuss  the  specific  items  of  information  that 
we  utilize  in  our  procedure,  and  how  these  entries  can  then  be  employed 
in  course  design.  (As  I  briefly  discuss  each  of  these  items,  you 
might  refer  to  the  handout  which  was  provided  to  you  earlier.  The 
second  page  of  your  handout  explains,  in  more  detail,  the  meaning 
of  each  item  that  I  discuss). 

An  extract  of  our  computer  print-out  begins  on  the  third  page 
of  the  handout.  As  you  can  see,  the  task  description  makes  up  a 
major  portion  of  it.  Bach  statement  has  associated  with  it  a  ten¬ 
digit  identification  number.  Ten  digits  were  selected  to  provide 
the  flexibility  for  as  much  hierarchical  structuring  as  we  thought 
necessary  to  adequately  reflect  the  requirements  of  the  relevant 
tasks. 

Bach  task  and  tasx  component  statement  is  cross-reference  in  the 
column  labeled  REP  with  a  doctrinal  publication  governing  this 
activity.  When  a  change  occurs  in  a  regulation,  the  task  can  be 
quickly  and  easily  updated  to  reflect  the  change. 
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The  next  field  in  our  printout  contains  a  description  of  the  en- 
abling  knowledge  required  to  perform  an  activity.  This  is  followed 
in  Column  A  by  a  code  which  indicates  if  there  is  an  enabling  skill 
requirement.  It  is  when  tasks  are  reduced  to  their  components  that 
specific  knowledges  and  skills  needed  to  carry  on  the  task  become 
evident.  These  knowledge  and  skill  entires  provide  information  to 
training  program  developers  of  the  intensive  practice  or  remediation 
which  may  be  required  at  certain  points  within  the  program. 

Judgments  of  the  importance  or  criticality  of  the  particular  tasks 
are  to  be  coded  and  listed  in  Column  B.  These  judgments,  along  with 
the  frequency  of  performance  information  that  will  be  recorded  in 
Column  F  enter  into  the  processes  of  selecting  tasks  for  training, 
and  determining  performance  standards  to  be  met  in  the  course.  Re¬ 
cording  this  information  and  displaying  it  utilizing  our  technique 
enables  training  developers  to  more  easily  identify  the  required 
tasks  for  training  and  to  derive  performance  standards.  These 
standards  would  then  be  recorded  in  Columns  G  and  H.  The  decisions 
made  for  selecting  training  tasks  are  to  be  noted  and  coded  in 
Column  D. 

The  last  page  of  your  handout  is  the  recording  form  that  we  use 
in  our  procedure.  The  task  description  statements  are  not  recorded 
on  this  form,  but  rather  on  ordinary  plain  white  paper.  Identifi¬ 
cation  numbers  are  then  assigned  to  them.  The  number  of  each  specific 
item  is  then  recorded  on  this  sheet  (the  recording  form)  and  the  addi¬ 
tional  information  entered  as  it  is  obtained. 

Now  I  would  like  to  concentrate  my  discussion  on  the  entries  in 
Column  C,  Course  Location,  and  Column  E,  Test  Item  Location.  In  our 
opinion,  it  is  of  little  value  to  apply  a  systematic  mechanism  to  the 
development  of  training  content  if  there  is  no  way  of  checking  to  see 
that  this  mechanism  is  being  utilized  properly.  We  have  set  up  a 
procedure  that  we  call  a  "discrepancy  analysis"  in  which  the  tasks 
selected  for  training  (the  training  objectives)  are  checked  against 
course  content.  In  a  similiar  manner,  the  test  items  are  recorded 
as  they  apply  to  each  of  the  tasks  or  task  components.  Each  of  every 
narrative  lesson  plan  and  each  item  in  every  test  is  examined  and  the 
location  recorded  next  to  the  appropriate  corresponding  task  or 
component.  One  can  then  come  up  with  lists  of  discrepancies  between 
training  objectives,  course  content,  and  test  items.  These  dis¬ 
crepancies  can  be  ultimately  resolved  by  the  training  program  and 
test  developers.  By  employing  this  discrepancy  analysis  procedure 
in  our  course  design  activities,  we  have  provided  a  quality  control 
checkpoint  during  curriculum  development.  It  is  in  this  regard  that 
the  print-out  document  serves  as  a  control  device  to  ensure  corres¬ 
pondence  between  the  job  and  the  content  of  the  course,  and  also 
between  the  course  content  and  the  test  item  coverage.  Thus,  we 
ultimately  arrive  at  job-related  training  and  performance-oriented 
examinations  of  student  capabilities. 

There  are  some  other  special  features  of  our  computer  program, 
which  might  be  of  interest  to  you.  When  one  makes  up  a  listings 
of  tasks  and  their  components,  there  are  often  times  several  separate 
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major  tasks  tbat  include  within  them  component  sequences  that  are 
identical.  In  order  to  aid  the  course  designer  as  he  works  with 
the  task  analysis  document,  each  of  the  steps  required  for  per¬ 
formance  of  a  major  task  should  be  recorded  in  the  context  in  which 
it  occurs.  There  is  a  routine  within  our  set  of  programs  which 
permits  the  computer  to  search  the  records  and  select  statements 
that  we  have  identified  in  a  certain  way,  and  then  insert  them 
within  the  desired  context.  This  technique  greatly  aids  the  work 
of  the  task  analysis. 

Thus,  curriculum  developers  provided  with  this  print-out  are 
given  a  device  which  is  designed  specifically  for  training  develop¬ 
ment  purposes  and,  in  addition,  is  convenient  to  use. 

It  is  our  belief  that  the  approach  we  have  taken  is  flexible 
enough  to  be  useful  to  a  variety  of  programs  within  the  CONARC 
training  system.  We  have  not  developed  any  panacea  for  systems 
engineering  a  training  program.  As  you  can  see,  much  work  is  re¬ 
quired  to  obtain  the  information  for  our  print-out,  and  to  perform 
the  discrepancy  analyses.  However,  we  feel  that  we  have  made  a 
modest  contribution  toward  systematically  and  conveniently  arranging 
the  information  necessary  for  adequate  course  design  and  development. 
We  hope  that  you  will  find  in  our  approach  some  applicability  to 
your  needs. 
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COURSE  STRUCTURE  RATIQNALS 


Dr.  Charles  0.  Gray 
U.  S,  Army  Engineer  School 
Fort  Belvoir,  Virginia 


GENERAL 

This  presentation  received  its  impetus  from  a  statement  found 
in  paragraph  17b,  CON  Reg  350-100-1  -  "Remember  at  all  times,  how¬ 
ever,  that  it  is  the  structuring  of  the  training  objectives  that 
is  important  not  the  rationale  used  in  arriving  at  that  structure." 
This  statement  when  taken  at  face  value  infers  that  the  rationale 
underlying  structure  is  of  little  significance.  It  is  the  opinion 
of  this  school  that  course  structure  effectiveness  from  the  learner*s 
viewpoint  has  a  relationship  to  rationale.  In  other  words  this 
school  is  hypothesizing  that  in  regards  to  learning  effectiveness 
indices  (grades)  an  optimum  course  structure  rationale  will  result  in 
a  higher  order  of  learning  effectiveness  and  that  an  optimum  course 
structure  rationale  can  be  identified  for  predetermined  types  of 
courses . 

While  these  hypotheses  have  not  been  subjected  to  complete 
scientific  testing,  considerable  thought  and  effort  have  been  ex¬ 
pended  in  arriving  at  basic  parameters  which  should  be  fundamental 
in  the  eventual  pursuance  of  statistically  proving  or  disproving 
the  hypotheses.  It  is  these  parameters  that  will  form  the  basis 
tor  these  remarks  and  provide  the  substance  for  the  following  ob¬ 
jectives  of  this  presentations 

a.  To  standardize  course  structure  components. 

b.  To  provide  a  basis  for  course  structure  rationale. 

These  objectives  dicate  discussing  the  following. 

a.  Systems  Engineering  Context 

b.  Course  Structure  Components 

c.  Course  Structure  Alternatives 
SYSTEMS  ENGINEERING  CONTEXT 

At  the  risk  of  being  too  simplistic  and  redundant,  the  position 
of  determining  course  structure  in  applying  the  basic  steps  of  the 
Systems  Engineering  of  Training  as  found  in  CON  Reg  350-100-1  is 
herein  reiterated.  We  know  fundamentally  that  the  course  structure 
determination  action  is  included  in  the  third  basic  step  of  the 
System-Training  Analysis.  This  step  requires  these  actions: 

a.  Identifying  the  job  conditions,  standards  and  supporting  skills, 
knowledges  and  attitudes. 

b.  Converting  the  job  requirements  to  training  objectives  and 
criteria. 

c.  Developing  the  course  structure. 

d.  Developing  a  course  evaluation  concept. 

COURSE  STRUCTURE  COMPONENTS 

Faced'  with  the  proliferation  of  labels  to  identify  the  components 
of  a  course,  this  school  has  adopted  a  hierarchy  of  components  which 
has  clarified  the  proliferation.  This  hierarchy  of  components  is 
very  simple.  It  consists  of  the  following  in  progressive  inclusive 
order:  lesson,  block,  segment  and  course. 

A  lesson  is  an  integral  package  of  learning  experiences  built 
upon  a  structure  providing  for  an  introduction,  presentation, 
application  and  check-up  which  is  designed  to  contribute  to  or 
tally  accomplish  the  training  objective  as  defined  on  a  Training 
Information  Sheet  .  The  subject  matter  content  of  a  lesson  consists 
of  one  or  more  learning  elements  listed  on  the  Learning  Analysis  Sheet 
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that  supports  a  Training  Analysis  Information  Sheet.  The  lesson 
title  is  synonomous  with  the  subject  identification  listed  under 
a  program  of  instruction  annex. 

A  block  is  an  identifier  of  one  or  more  related  lessons  which 
is  defined  for  the  purpose  of  providing  first  line  and  intermediate 
levels  of  supervision  with  a  framework  for  quality  control  assess¬ 
ment. 

A  segment  is  an  identifier  of  one  or  more  related  blocks  which 
is  defines!  for  the  purpose  of  providing  top  management  with  a  frame¬ 
work  for  quality  control  assessment.  Segments  normally  have  a  direct 
correlation  with  instructional  organizational  elements. 

A  course  is  the  sum  of  all  segments  heretofore  identified. 

COURSE  STRUCTURE  ALTERNATIVES 

For  the  purpose  of  pursing  course  rationale,  the  structure 
alternatives  pursued  in  this  section  give  direction  to  the  seg¬ 
ment  identifiers  of  a  course.  This  section  is  essentially  related 
to  the  hypothesis  stated  in  paragraph  1  -  that  an  optimum  course 
structure  rationale  can  be  identified  for  predetermined  types  of 
courses.  Preliminary  effort  related  to  this  hypothesis  ha?  con¬ 
sisted  of  developing  a  course  types  matrix  using  the  factors  of 
student  experience  and  learning,  the  application  of  this  martrix 
to  the  local  curriculum,  the  identification  of  a  cariety  of  course 
structure  alternatives,  and  of  developing  a  course  type-structure 
martrix  using  the  factors  of  course  type  and  structure  alternatives. 

The  course  types  matrix  contained  in  Figure  1  incorporates  the 
following  defined  factors. 

a.  Student  Experience  Factors.  Every  learning  situation  must 
consider  what  skills  and  knowledges  the  learner  brings  with 
him  when  he  enters  the  learning  enviroment.  These  factors 
for  the  purpose  of  this  matrix  are  broken  down  into  three 
discrete  levels  as  follows: 

(1)  Entry  Training  Only  (BCT/ROTC).  This  factor  identifies 
the  training  population  whose  Army  experience  has  been 
limited  to  either  Basic  Combat  Training  for  enlisted 
personnel  or  Reserve  Officer  Training  Corps  participation 
for  officer  personnel. 

(2)  Advanced  Individual  Training  (AIT).  The  training  population 
assigned  to  this  factor  have  completed  the  entry  training 
described  in  (1)  above  and  an  additional  MOS-producing 
course.  They  have  had  no  unit  experience. 

(3)  Entry,  AIT  and  Unit  Experience.  This  factor  encompasses 
the  previous  two  factors  and  includes  some  quantity  of 
unit  experience. 

b.  Learning  Factors.  What  students  learn  in  a  training  and/or 
education  program  may  be  defined  in  a  variety  of  ways.  Today's 
learning  enviroment  places  a  premium  on  skill  developnment 

to  job  performance  vs  the  past  emphasis  on  academic  skills  and 
knowledges.  For  the  purpose  of  this  matrix  skill  development 
is  broken  into  the  following  two  categories: 
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(1)  Soft  Skills.  These  are  job-related  skills  involving 
actions  affecting  primarily  people  and  paper,  e.g., 
inspecting  troops,  supervising  office  personnel,  con¬ 
ducting  studies .preparing  maintenance  reports,  pre¬ 
paring  efficiency  reports,  designing  bridge  structures. 

(2)  Hard  Skills.  These  are  job-related  skills  involving 
actions  to  physical  things.,  e.g.,  adjusting  rifle 
headspace,  timing  an  engine,  operating  a  bulldozer, 
conducting  preventive  maintenance  on  an  aiming  circle. 


COURSE  TYPES  MATRIX 


STUD  ENT 

EXPERIENCE^.  LEARNING 

FACTORS  FACTORS 

SOFT  SKILLS 

HARD  SKILLS 

Entry  Training  Only 
(BCT/ROTC) 

1 

2 

AIT 

3 

4 

Entry,  AIT  and 

Unit  Experience 

5 

6 

Figure  1 


The  application  of  this  matrix  to  the  local  curriculum  has  resulted 
in  the  following  typology. 

COURSES 

4-5-C20  -  Engineer  Officer  Basic  Course 
4-5-C1  -  Engineer  Officer  Basic  (Nonresident/Resident) 

35E20  -  Special  Electrical/Electronic  Device  Repair 

41B20  -  Topographic  Instrument  Repair 

41K20  -  Reproduction  Equipment  Repair 

51L20  -  Refrigeration  Equipment  Repair 

52B20  -  Mobile  Electric  Power  Generation 

52C20  -  Power  Pack  Specialist 

62B20  -  Engineer  Equipment  Maintenance 

62C20  -  Engineer  Missile  Equipment  Maintenance 

( 


TYPE 

1 

2 
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COURSES 


81B20  -  Construction  Drafting 

81C20  -  Cartographic  Drafting 

81D20  -  Map  Compiling 

82E20  -  Topographic  Computing 

12E20  -  Atomic  Demolitions  Enlisted 

51G20  m  Soils  Analysis 
82B20  -  Construction  Surveying 

83D20  -  Process  Photography 

83B20  -  Lithographic  Platemaking 

83F20  -  Offset  Press  Operations 

4A-F1  -  Engineer  Construction  Officer 

4L-0663  -  Engineer  Equipment  Officer 
4A-7130  -  Facilities  Engineering  Management 
4M-7915  -  Topographic  Engineer  Officer 

4E-F1  -  Atomic  Demolitions  Munitions  Officer 

2E-F39  -  Nuclear  and  Chemical  Target  Analysis 

030-F1  •  Special  Forces  Engineer  Training 

35E30  -  Senior  Special  Electrical/Electronic  Device 

Repair 

52B30  -  Mobile  Electric  Power  Generation  (Precise) 

52D20  -  Gas  Turbine  Generator  Repair 

62B30  -  Engineer  Equipment  Repair 

62C30  -  Engineer  Missile  Equipment  Repair 

81D30  -  Photogrammetric  Compilation 

612-F1  -  Mobile  Assault  Bridge/Ferry  Maintenance 

652-F3  -  Utility  Element  Maintenance  MUST 

4-5-C22  •  Engineer  Officer  Advanced 

4-.5-G23  -  Engineer  Officer  Advanced  (Nonresident/ 

Resident) 

2E-F27  -  Engineer  Staff  Officer  Refresher 

4-5-C8  -  Engineer  Field  Grade  Officer  Refresher 

7K-F2  -  Diaster  Recovery 

710~F1  -  Engineer  Noncommissioned  Officer 

4-5-C30  -  Engineer  Graphics  Warrant  Officer 
Intermediate 

7K-F10/440-FI  -  Fallout  Shelter  Analysis 
4A-F4  -  Engineer  Construction  Contracting 

4M-F1/412-F2  -  Advanced  Geodetic  Surveyor 
4L621A/612F2  -  Engineer  Equipment  Repair  Technician 
Bngineer  N00  Basic 


While  the  range  of  course  structure  alternatives  is  wide, 
this  school  has  identified  the  following  eight  as  representative. 
Associated  with  each  alternative  is  an  example  drawn  from  the 
local  curriculum. 


ALTERNATIVE 


EXAMPLE 


1 .  Hardware 


2.  Tasks 


3.  Functions 


4.  Environment 

5.  Proficiency  Levels 

6 .  Knowledge  Areas 


7.  Duty  Areas 


8.  Systems 


a.  Surveying  Rods 

b.  Abney  Hand  Level 

c.  Dumpy  Level 

d.  Transit 

e.  Telescopic  Alidade 

a.  Maintenance  of  gasoline  and  diesel  engines 

b.  Maintenance  of  generator  body,  cab  and 
frame  assembly 

c.  Maintenance  of  generator  electrical 
assembly 

d.  Maintenance  of  generator  control  panels 
and  housing  cubicles 

e.  Maintenance  of  generator  governing  and 
AC  voltage  regulating  systems 

a .  Supply 

b.  Administration 

c .  Maintenance 

d.  Research  and  Development 

a.  Construction  Engineering  Battalion 

b.  Combat  Engineering  Battalion 

c.  Civil  Works  District 

a.  Basic  Cartographic  Drafting 

b.  Advanced  Cartographic  Drafting 

e.  Mathematics 

b.  General  Subjects 

c.  Map  Reading 

d.  Methods  of  Instruction 

a.  Generator  Set  Operation 

b.  Generator  and  Site  Selection 

c.  Generator  Set  Installation 

d.  Organizational  Maintenance  of  Air  Cooled 
Gasoline  Engines,  Generator  Sets  AC  and 
DC,  and  Control  Panels 

a.  Manually  Controlled  Fixed  Power  Plant 

b.  Automatically  Controlled  Fixed  Power  Plant 


With  the  previously  identified  course  typology  and  structure 
alternatives,  a  course  type  «•  structure  matrix  has  been  developed 
with  the  entries  thereon  being  purely  judgmental  at  this  point  in 
time.  This  natrix  is  portrayed  in  Figure  2. 
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CCMMERCIAL  APPROACH  TO  SYSTEMS  ENGINEERING 


Dr.  William  G.  Hoyt 

Systems  Development  Corporation 

Santa  Monica,  California 


This  paper  covers  the  systems  engineering  of  training  of  a  new 
major  weapons  system.  Safeguard,  by  the  System  Development  Corpor¬ 
ation  (SDC)  in  accordance  with  CONARC  Regulation  350-100-1,  Systems 
Engineering  of  Training. 

The  project  included: 

(1)  TASA  Data  collection  at  six  manufacturers  plant  locations 
by  teams  of  SDC  and  manufacturer  personnel. 

(2)  Data  Base  Development  utilizing  a  360-67  computer  and  the 
SDC  Data  Management  System  computer  programs. 

(3)  Job  structure  analysis  to  determine  recommended  duty 
positions  and  M0S*s. 

(4)  POI  development. 

The  problems  resolved  included: 

Selecting  unique  tasks  for  training  from  a  large  data  base; 
controlling  the  step  processes  required  in  instituting  and  up¬ 
dating  the  system  engineering  process  by  means  of  the  task  string 
listing;  and  the  development  of  POI  in  the  absence  of  a  defined 
and  approved  job  structure  by  modular  construction  of  a  baseline 
POI. 

The  computer  is  used  in  the  process  of  establishing  the  data  bank 
selecting  tasks  for  training  and  reporting  and  controlling  the  POI 
development  as  it  progresses.  The  computer  is  also  used  to  establish 
the  data  audit  trace  which  is  a  complete  record  of  the  systems  eng¬ 
ineering  process,  it  shows  the  training  decision  for  every  task  in 
the  data  bank,  the  associated  job  Task  Data  Cards,  Training  Analysis 
Information  sheet,  each  PIO  the  task  is  in  and  whether  a  trainer 
evaulation  has  been  made. 

At  this  time,  the  training  analysis  has  been  conducted  on  over 
6500  tasks.  Selected  for  training  are  approximately  1,050  unique 
tasks  and  600  filler  tasks.  The  training  analysis  for  the  over 
10,000  tasks  expected  in  the  data  bank  will  be  completed  by 
July  1972. 

The  results  of  the  TASA  project  will  be  the  training  requirements 
for  resident  and  on-site  training  to  include  both  on-line  and  off-line 
maintenance  and  maintenance  operations. 


SYSTEMS  ENGINEERING  WORKSHOP  OBSERVATIONS 


Dr.  Robert  M.  Gagne* 

Florida  State  University 
Tallahassee,  Florida  20380 


I  have  listened  to  your  presentations  with  an  enormous  degree 
of  interest  and  1  think  they  leave  little  doubt  that  you  are  all 
committed  to,  and  are  also  pursuing  vigorously,  the  systems  eng- 
ineering  approaches  to  designing  instruction  in  your  courses. 

Really,  there  are  many  things  that  I  could  comment  on  and  I  have 
little  trouble,  I  guess,  deciding  what  are  the  most  important 
things  to  say.  Obviously  that's  what  I  should  be  doing.  I  guess 
one  impression  that  I  have  that's  sort  of  interesting  and  perhaps 
something  that  you  might  attempt  to  use  -  there  is  considerable 
diversity  in  the  techniques  that  you  have  reported.  There’s  a 
great  deal  of  commonality  in  the  aims  that  these  techniques  have, 
but,  at  the  same  time,  a  great  deal  of  diversity.  For  example, 
how  does  one  describe  objectives  most  conveniently?  Or  how  does 
one  decide  upon  the  different  categories  of  learning  outcomes?  We 
had  several  techniques  mentioned.  These  among  others.  Or  how 
does  one  go  about  the  planning  of  lessons,  the  sequencing  of  skills 
to  be  learned?  Or  how  does  one  design  proper  measures  or  use 
proper  assessment  techniques? 

It  seems  to  me  you  are  coming  to  a  point  where  you  might  want 
to  aim  for  optimization  of  these  techniques,  rather  than  for  great** 
er  diversity  of  them.  Now,  in  this  respect,  I  think  you  differ 
from  the  civilian  There  is  a  similar  diversity  in  the 

civilian  sector,  as  I  am  sure  you  are  aware.  You  can  find  schools 
that  are  trying  individualized  instruction  and,  in  fact,  they  are 
trying  all  kinds  of  individualized  instruction.  You  can  find 
schools  which  say,  "We  are  stating  performance  objectives."  And 
you'll  find  great  diversity  in  that  and  how  they  do  it  and  so  on. 

And  you  can  find  nongraded  schools  and  graded  schools  and  traditional 
schools,  I  suppose,  although  it  would  be  difficult  to  say  just  what 
a  traditional  school  is.  There  is  great  diversity,  also,  in  the 
civilian  sector.  Perhaps  you  have  a  difference  here,  perhaps  you 
have  an  opportunity  which  is  better  than  exists  in  the  civilian 
case,  of  trying  to  achieve  an  potimum  set  of  methods.  Perhaps 
you  could  adopt  this  as  a  goal.  You  may  be  able  to  work  towards 
the  optimizing  of  these  techniques  that  you  have  described  and 
overcome  the  disadvantages  of  diversity.  And  there  are  disadvantages 
to  diversity,  it  seems  to  me.  A  workshop  like  this,  of  course, 
is  one  step  in  that  direction.  Here  you  exchange  information.  You 
find  people  know  certain  kinds  of  techniques  that  fill  in  a  gap  that 
you  have  been  aware  of  and  that  you  have  been  struggling  with.  You 
can  obtain  this  information.  And  exchanging  this  information  is  one 
way,  or  one  step,  in  that  direction.  I  would  think  that  another 
possibility  would  be  you  might  try  to  establish  some  task  forces, 

I  would  assume  they  would  have  to  be  inter-service,  which  try  to 
achieve  statements  of  optimized  methods,  of  going  about  these 
various  techniques  that  make  up  systems  engineering.  There  is  also 
the  role,  as  I  am  sure  you  realize,  of  a  higher  headquarters  in  this. 
Now,  I  think  that  certainly  one  needs  to  be  aware  that  there  is  an 
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advantage  to  having  eommand  authority  in  this  kind  of  situation. 

On  the  other  hand,  I  think  that  such  command  authority  should  be 
very  much  aware  of  the  desirable  aspects  of  diversity,  so  there 
are  ways,  it  seems  to  me,  of  achieving  optimized  methods  and  an 
optimal  system  and  there  are  ways  that  are  good  and  there  are 
ways  that  are  not  so  good,  because  certainly,  I  think  any  higher 
headquarters  is  not  interested  in  restraining  the  creative  efforts 
of  its  subordinate  schools  or  instructors  or  training  supervisors 
or  whatever.  For  example,  if  a  higher  headquarters  were  to  say, 

"What  we  would  like  to  have  is  individualized  instruction,"  it 
could  proceed,  it  seems  to  me,  and  this  is  probably  kind  of  wild 
and  I  don’t  suppose  that  would  happen,  but  it  could  proceed  by 
saying,  "O.K.,  Schools,  we  are  going  to  supply  you  with  students 
now  at  any  time.  They  might  report  on  any  date  and  they  are 
supposed  to  start  their  instruction  immediately  and  they  are 
supposed  to  continue  their  instruction  until  they  have  won  certain 
criteria  which  you  have  established."  Well,  it  seems  to  me,  that 
kind  of  a  directive  would,  in  fact,  force  people  to  use  individual* 
ized  instruction.  It  wouldn’t  tell  them  how  to  do  it,  you  see,  but 
it  would  say  that  has  to  be  done  because  every  individual  has  to 
be  treated  as  an  individual,  even  to  the  extent  of  when  he  reports 
to  the  school  and  when  he  reports  to  become  trained  in  whatever 
speciality  it  is.  So  that  may  be  an  example  of  the  kind  of  directive 
that  would,  I  think,  bring  about  a  change.  At  the  same  time,  it 
would  not  be  restraining  on  the  particular  methods,  particular  ways, 
an  individual  school’s  directors  and  so  on  would  use  to  bring  it 
about,  but  it  would,  I  think,  assure  that  this  aim  would,  in  fact, 
accomplished.  That  is  just  an  example.  It  may  be  not  a  very 
practical  one. 

One  other  thing  that  I  thought  I  ought  to  comment  on  was  that 
it  is  apparent  that  you  are  coming  up  against  the  hard  question  of 
soft  skills.  I  think  there’s  a  very  important  distinction  to  be 
made  between  hard  skills  and  soft  skills,  although  for  some  purposes 
the  distinction  between  hard  and  soft  might  be  appropriate  in  terms 
of  whether  or  not  hardware  is  involved.  I  don’t  think  that,  that 
distinction  fully  captures  perhaps,  the  importance  of  what  is  meant 
in  various  schools  and  in  various  situations.  Actually,  it  may  be 
that  soft  skills  are  not  really  skills  and  that  may  be  the  problem. 

What  about  the  distinction  between  hard  skills  and  soft  skills? 

Let  me  try  a  definition.  I  would  say,  hard  skills  are  those  capable 
of  these  which  can  be  assessed  as  learning  outcomes  by  tasks  which 
make  possible  the  decision  that  they  have  or  have  not  been  learned. 

Or  perhaps,  that  they  have  or  have  not  been  learned  to  some  specified 
criteria.  But  soft  skills  are  not  like  that.  I  think  soft  skills 
are  those  capabilities  whose  assessment  yields  evidence  of  more  or 
less,  of  better  or  worse,  and  1  don’t  know  whether  there  is,  in  fact, 
a  specifiable  standard  or  criteria  against  which  these  can  be  assessed. 
For  example,  an  individual  has  a  more  or  less  positive  attitude  to* 
wards,  let  us  say,  some  desirable  quality  like  dependability.  Or  he 
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behaves  in  a  leadership  situation  more  or  less  well.  Some  are 
better  than  others,  and  it  isn*t  possible  to  say,  '’This  is  the 
criteria.”  Now,  you  will  find  on  this  handout  a  set  of  categories 
which,  it  seems  to  me,  illustrate  some  of  these  differences  that 
I  am  trying,  at  this  point,  to  emphasize.  If  you  see,  under 
"Intellectual  Skills"  ...  look  at  that  first.  These  are  the  hard 
skills.  As  far  as  I  can  see  these  are  the  ones  about  which  you 
can  say  you  know,  we  can  formulate  specific  objectives  and  say  the 
individual  has  learned  them  or  he  has  not  and  that*s  it. 

Now,  motor  skills  are  often  treated  that  way  and  so  they  pro¬ 
bably  belong  in  the  hard  skill  category,  too,  although  even  to 
put  them  there  requires  a  little  bit  of  adjustment.  Motor  skills, 
as  I  think  is  well  known,  continue  their  improvement  with  practice 
over  long  periods  of  time.  And  so,  what  we  are  doing  when  we  put 
motor  skills  in  the  category  of  hard  skills,  is,  in  fact,  we  are 
saying,  "Oh,  well,  he  does  it  well  enough  and  I  can  specify  a  crit¬ 
eria  here,  and  if  he  does  it  that  well,  O.K."  But  I  suppose  we 
should  have  this  mental  reservation  about  that,  and  If  1  provided 
him  practice  on  this  motor  skill  he  could  get  better  and  better  at 
it  so  far  as  anyone  knows  over  a  very  long  period  of  time.  However, 
that  is  incidental.  1  think  motor  skills  do  belong  in  that  cate¬ 
gory. 

Now,  what  about  verbal  information?  Well,  that's  a  puzzle.  You 
think  about  it  as  a  hard  skill  and  again,  I  think,  it  is  -  but  only 
if  again  you  adopt  a  kind  of  an  arbitrary,  perhaps,  but  a  criteria 
which  satisfies  you.  Now  think  ...  here  is  an  example.  Suppose  you 
say,  I  hav  a  course.  I  don't  know  whether  you  deal  with  these  kinds 
of  courses  or  not,  but  suppose  you  say,  "I  have  a  course  in  history 
and  I  am  teaching  these  students  about  the  American  Revoltuion."  All 
right,  "I  want  to  teach  them  about  the  American  Revolution."  But 
what?  How  much  do  you  want  to  teach  them  about  the  American  Revolu¬ 
tion? 

Well,  perhaps  you  can  say,  "All  right.  I  want  them  to  be  able  to 
state  three  reasons  for  the  origin  of  the  American  Revolution."  All 
right,  if  that  is  a  satisfactory  standard,  O.K.  But  why  three?  Why 
not  one?  Why  not  2?  Why  not  32?  You  see,  it's  very  difficult  to 
set  a  standard  or  a  criteria  for  the  learning  of  verbal  information. 
We  have  to  adopt  rather  arbitrary  standards,  so  I  put  verbal  infor¬ 
mation  in  a  rather  questionable  category,  as  to  whether  or  not  it's 
a  hard  skill.  I  do  that  because  I  recognize  that  knowledge.,,  you 
see,  the  accumlation  of  verbal  information  is  generally  referred  to 
as  knowledge  and  knowledge,  you  know,  just  accumulates  for  a  long, 
long  time  and  it  gets  added  to  in  many  ways.  Now,  how  much  infor¬ 
mation  does  anybody  need?  Is  a  very  difficult  thing  to  specify  and 
1  am  not  sure  that  it  belongs  in  the  category  of  hard  skills,  al¬ 
though  I  think  that  there  are  many  instances  that  you  have  mentioned 
in  the  case  of  your  epecific  aims  for  your  cources  where,  in  fact, 
it  can  be  treated  that  way. 
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Now,  we  come  to  some  things,  Let's  consider  the  other  two 
categories  that  are  here;  attitudes  and  cognative  strategies. 

Now  here  is  where  you  really  get  into  trouble.  I  do  not  see, 
at  the  moment,  how  these  can  be  handled  as  hard  skills  or  how 
they  can  be  system  engineered  by  means  of  the  same  methods  that 
you  are  using  for  others.  I  believe  that  we  need  ...  I  say, 

"we”  -  I  am  just  as  concerned  about  this  as  you  are  -  I  think 
that  we  need  new  methods  to  handle  these  things  and  I  think  that 
if  you  try  to  force  these  kinds  of  things  into  the  hard  skill 
use  with  the  hard  skill  techniques  that  you  will  be  unhappy. 

We'll  all  be  unhappy.  It  won#t  work.  We  need  a  larger  technology 
here  to  handle  these  categories. 

Let  me  say  a  little  more  about  attitudes.  Attitudes  are  very 
important  things.  Some  of  you  have  mentioned  this  in  some  of  the 
discussion  previously.  You  want  the  trainees  that  you  deal  with, 
not  just  to  have  rather  specific  attitudes  about  being  careful 
to  turn  off  the  equipment,  you  want  them  to  have  a  positive  atti¬ 
tude  toward  their  job.  You  want  them  to  have  positive  attitudes 
toward  their  branch  of  service.  You  want  them  to  have  positive 
attitudes  toward  the  kind  of  career  that  they  are  in.  How  are 
these  things  established?  Well,  I  don't  think,  you  know,  that  the 
methods  that  have  been  described  will  handle  this,  particularly 
because,  it  seems  to  me,  that  one  of  the  requirements  for  learning 
attitudes,  1  think,  is  a  human  model.  And  again,  this  has  been 
mintioned  this  morning.  It  seems  to  me  that  however  much  infor¬ 
mation  we  provide  and  however  carefully  we  sequence  or  arrange  the 
learning  of  subordinate  skills,  we  will  not  achieve  this  matter 
of  establishing  or  changing  attitudes  which  is  a  very  important 
part  of  what  you*re  dealing  with  when  you  talk  about  soft  skills. 

1  think  it  requires  a  human  being.  It  requires  a  human  model. 
Certainly  the  evidence  shows  one  that,  you  know,  that  the  things 
that  don't  work  in  teaching  attitudes  are  the  providing  of  information 
or  the  speaking  to  people  somewhat  as  I  am  doing  to  you.  You  know, 

I  can  say  to  you,  "Look!  You  should  always  be  honest,"  or  "You 
should  always  be  prompt,"  or  "You  should  always  be  a  leader,"  and 
you  know  that  that  is  ineffective.  That  does  not  work  and,  Goodness 
knows,  the  Army  has  had  plenty  of  examples  of  that  during,  per¬ 
haps,  many  years  of  it* s  existence  -  the  Army  and  the  other  services 
as  well.  What  does  work,  according  to  the  evidence  that  I  see, 
is  the  use  of  a  human  model.  The  individual...  I  don't  know  what 
happens.  You  know,  1  don't  know  how  to  explain  it,  He  identifies, 
perhaps  with  this  other  human  being  and  he  models  his  behavior,  it 
you  want,  he  models  his  in  such  a  way  that  he  acquires  a  general 
tendency  to  respond  positively  or  negatively,  or  more  or  less 
positively  or  negatively,  toward  some  object  or  event  or  person. 

And  that's  the  kind  of  thing  that  seems  to  me  to  be  critical  for 
the  establishment  of  an  attitude.  Another  factor  which  is,  I 
think,  equally  critical  is  the  personal  experience  of  success. 

You  see,  will  this  individual  like  the  job  that  he  has  chosen? 
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Will  he  like  to  do  the  kinds  of  things  that  he  has  chosen  to  learn 
to  do?  Well,  one  factor  in  this  is  his  personal  experience  of 
success.  Now,  several  of  these  things  indicate,  you  see,  the  im¬ 
portance  of  the  instructor  ...  of  the  instructor  or  other  leaders 
in  the  military  community  and  1  don’t  think  that  that  is  a  lot 
of  baloney.  I  think  that  that’s  true  ..  that  there  is  human 
modeling  and  that  this  is  a  very  important  part  of  what  must  be 
attended  to  if  we  want  to  establish  attitudes.  This  is  a  general 
category  that  1  give  to  this  kind  of  thing. 

Now  cognative  strategies  ...  you  see,  I  have  been  relieved 
to  find  out  that  much  of  what  you  have  been  talking  about  when 
you  talk  about  leadership  is  the  ability  to  formulate  and  think 
out  the  solution  of  problems.  That’s  really  what  it  is.  In  other 
words,  you  are  trying  to  train  your  men  ...  your  officers  to  think 
That’s  really  what  it  is  about.  You  want  them  to  solve  certain 
kinds  of  problems.  And  we  had  this  morning  a  description  of  the 

kinds  of  things  that  occue  in  these  soft  skill  courses. 

You  are  trying  to  teach  officer  ...  well,  how  do  they  go  about 
taking  an  enemy  position?  Well  now,  that  can’t  be  taught  as  a 
skill.  You  are  asking  an  officer  to  solve  a  normal  problem  in 
that  situation  and  so,  1  think,  again,  here  is  another  area  that 
may  require  an  expansion  of  your  technology,  of  my  technology,  of 
anybody’s  technology  in  order  to  handle  this.  How  do  we  handle 
this  now?  Well,  the  only  way  I  know  to  handle  it  is  to  provide 
opportunities  for  practice  in  a  variety  of  problem  situations  ... 
you  know,  simulations,  or  whatever.  That’s  what  we  do:  that’s 
what  people  in  the  schools  do.  Nobody  knows  how  to  do  it  any  better 

than  that.  Maybe  some  day  we  will,  but  we  don’t  now  know  how  to  do 

this.  Now,  of  course,  there  are  subordinate  skills  in  this.  If, 
in  fact,  you  are  trying  to  teach  the  individual  to  solve  a  problem 
having  to  do  with  the  letting  of  blood  or  the  use  of  a  tourniquet, 
then  obviously,  there  are  some  simple  ordinate  skills  that  can  be 
specified.  He  has  to  know  what  a  tourniquet  is  and  how  to  use  one, 
and  things  of  that  sort,  but  these  do  not,  in  themselves,  allow  him 
to  solve  this  problem.  You  are  presenting  him  with  a  novel  problem¬ 
solving  situation,  and  you  want  him  to  bring  to  bear  on  that  sit¬ 
uation  varieties  of  knowledge  from  all  fields  in  whatever  brains 
he  has,  you  know,  whatever  creative,  ingenious  kinds  of  thinking 
operations  he  has.  This  is  what  I  mean  by  cognate  of  strategies. 

So,  how  do  you  go  about  the  instruction  cognative  strategies?  And 
this  is  very  important  in  the  soft  skills  area.  Well,  you  must 
do  it,  I  think,  by  providing  these  opportunities  for  solving  novel 
problems.  I  wish  that  there  were  a  way  that  could  be  specified. 

You  see,  having  analyzed  that  situation,  the  answer  to  that  is,  in 
contrast  to  what  can  be  done  with  intellectual  skills,  we  don*t 
know  how  to  analyze  that  situation.  We  simply  do  not  have  the 
psychological  knowledge  that  enables  us  to  say,  "What  are  the 
components  of  good  thinking?"  We  don’t  know  that  and  I  think  that 
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that  means,  "Sure  there  are  limitations."  Practically  what  it 
means  is  you  have  to  put  people  in  as  realistic  situations  as  you 
can  in  order  to  give  them  opportunities  to  be  challenged  to  bring 
to  bear  on  those  situations  the  intellectual  resources  that  they 
have  and  that  get  developed  in  that  particular  way.  Now,  speak¬ 
ing  of  that,  of  course,  it  would  seem  to  me  doubtful,  you  see,  that 
you  can  develop  these  in  a  single  course.  It  doesn't  work  that 
way,  I  think,  it  takes  time  and  it  takes  many  opportunities  for 
problem-solving  and  thinking  and  these  things  get  better.  Some¬ 
body  said,  as  the  individual  gets  more  mature,  he  gets  better  at 
it,  so  I  doubt  whether  it  can  be  developed  in  a  single  course. 

You  know,  that's  true  in  the  civilian  sector,  too.  You  cannot 
give  a  course  in  how  to  think.  There  is  no  such  thing.  It's 
ridiculous  to  think  that  you  can.  Again,  you  can  establish  cer¬ 
tain  subordinate  skills  that  contribute  to  it.  Yes,  but  you 
really  cannot  give  a  six-week  course  or  a  two-week  course  in  how 
to  think.  It  takes  experience.  It  takes  a  lot  of  experience  in 
problem-solving  situations  to  get  the  individuals  to  be  able  to 
think  well  and  this  is,  I  think,  what  the  aim  of  such  soft  skill 
courses  might  be. 

So,  I  would  suggest  two  different  principles  here  I  think  you 
may  find  ...  that  ...  well,  at  least,  you  might  keep  them  in  mind 
because,  at  the  present  time,  this  is  the  way  it  seems  to  me. 

One  is  that  precise,  analytic  specification  and  measurement 
techniques  should  be  used  for  hard  skills.  Hard  skills  can  be 
and  should  be,  treated  as  hard  skills  -  not  as  soft  skills,  O.K. 

The  second  principle  is  precise,  analytic  specification  and 
measurement  techniques  are  not  suitable  for  soft  skills.  They 
are  probably  inappropriate  and  a  different  set  of  system  engineer¬ 
ing  techniques  may  be  needed.  The  main  thing,  I  believe,  at  this 
point,  and  certainly  your  presentations  have  caused  me  to  think 
about  these  things  very  deeply  ...  the  main  thing  it  seems  to  me 
probable  is  that  the  worst  mistake  to  make  is  to  mix  them  up.  You 
deal  with  the  hard  skills  in  one  way  and  you  can  ensure  their 
mastery  and,  I  believe  that  that  should  be  done.  Now,  I  think  back 
to  my  experiences  in  the  Air  Force  twenty  years  ago,  say,  and  we 
were  dealing  in  a  certain  sense  with  hard  skills  and  our  observa¬ 
tions  were  they  were  being  taught  to  a  large  extent  as  soft  skills 
and  that's  not  good.  We  thought  that  then.  So,  you  know,  I  am  de¬ 
lighted  to  know  that  there  has  been  such  an  enormous  amount  of 
acceptance  of  the  idea  that  hard  skills  can  be  treated  as  hard 
skills  and  they  can  be  analyzed  as  hard  skills  and  they  can  be 
taught  as  hard  skills  and  measured  as  hard  skills.  And  that's  a 
bad  mistake,  to  make,  I  think,  to  treat  them  as  soft  skills.  On 
the  other  hand,  I  think  that  the  distinction  is  very  important 
and  I  sort  of  doubt  whether  you  will  find  complete  success  or 
satisfaction  in  trying  to  teach  soft  skills  like  hard  skills. 
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I  think  you  need  a  larger  technology  and  one  that,  you  know, 
is  beyond  the  framework  most  of  which  you  have  emphasized  in  the 
various  presentations  that  1  have  heard. 

I  don’t  want  us  to  forget  that  the  human  being  is  an  enor¬ 
mously  flexible  organism  that  he  can  think  and  he  can  be 
stimulated  to  be  ingenious  and  creative  and  this,  after  all,  we 
don't  want  to  lose  these.  We  don’t  want  to  restrain  him  in  us¬ 
ing  these  capabilities  that  he  has  and  that  he  is  able  to  develop. 
So,  by  all  means,  we  should  give  him  the  fundamental  capabilities 
that  he  needs.  These  are  the  hard  skills  but  we  should  be  careful, 
I  think,  to  also  give  him  the  opportunities  to  develop  these  other 
things  which  we  call  soft  skills. 
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SYSTEMS  ENGINEERING  WORKSHOP  SUMMARY 

Although  time  will  not  permit  another  systems  engineering  Pig 
fable,  I  do  feel  it  necessary  to  recognize  Mr.  John  Myers,  Mr. 

Harris,  Sgts  Roff  and  Roby  of  the  Southeastern  Signal  School  for 
their  contributions  in  making  our  Workshop  a  Success. 

Now  in  developing  a  summary  of  the  Systems  Engineering  Work¬ 
shop  we  must  identify  the  objectives  established  and  the  level  of 
success  achieved.  Extensive  interaction  and  sharing  of  Systems 
Engineering  philosophy  procedures  and  techniques  were  our  stated 
objectives.  Our  purpose  was  directed  at  enriching  and  purifying 
the  systems  engineering  function  within  CONARC  in  an  attempt  to 
fully  capitalize  the  potential  available.  Our  workshop  opened 
with  presentations  from  our  sister  services  identifying  the  status 
of  the  systems  engineering  function  and  its  future  directions  within 
their  service.  It  was  apparent  that  each  of  the  military  services 
have  made  significant  strides  in  developing  the  function  and,  in¬ 
cluding  CONARC,  is  now  ready  to  expand  their  systems  engineering 
horizone.  We  are  in  complete  agreement  as  to  the  great  potential 
systems  engineering  possesses  in  dealing  with  individualization  of 
instruction,  methods  and  media  utilization  and  the  total  integration 
of  the  instructional  process.  And,  since  we  were  all  at  the  same 
basic  levels  the  need  for  developing  continuous  channels  of  cotnn- 
unications  for  sharing  our  developing  systems  engineering  functions 
between  the  services  was  identified.  We  are  indeed  grateful  to 
LTC  Godfrey  of  the  Marine  Corps,  Dr.  Havens  of  the  Navy,  and  Mr. 

Neale  of  the  Air  Force  for  -  through  them,  we  have  opened  these 
channels  of  communication,  which  will  undoubtedly  prove  mutually 
benef ic ial. 

Having  knowledge  of  the  new  directions  in  curriculum  development, 
research  was  essential  if  we  in  the  military  were  to  achieve  our  stated 
objectives.  Dr.  Gagne  of  Florida  State  University  provided  our  work¬ 
shop  with  a  stiuulating  overview  of  the  research  being  conducted  which 
will  certainly  influence  the  future  directions  of  systems  engineering 
development  within  CONARC.  Criterion  referenced  assessments  in  the 
development  of  systems  engineered  courses,  criterion  referenced 
evaluation  techniques  and  the  direction  towards  accountability  tech¬ 
niques  in  determining  the  attainment  of  goals  are  some  of  the  futur¬ 
istic  outlooks. 

Our  workshop  developed  around  those  major  areas  of  concern  as 
identified  by  our  participants,  and  the  presentations  of  some  of  the 
solutions  developed  by  individual  CONARC  elements.  Resolution  of 
our  manpower  problems  and  revision  of  our  guiding  regulation  were 
not  considered  although  universally  addressed  as  problems.  These 
two  problems  are  in  the  process  of  resolution  by  the  CONARC  staff. 

The  problems  that  were  considered  and  developed  within  the  workshop 
primarily  related  to  the  unique  procedures  and  techniques  developed 
by  various  CONARC  elements,  industry  and  the  civilian  community  to 
solve  these  problems.  Methods  and  Media  selection  and  individuali¬ 
zation  of  instruction  were  two  of  the  main  problems  discussed. 


As  we  all  know,  the  systems  engineering  function  involves  a 
myriad  number  of  interrelated  activities.  Our  workshop  maximized 
the  specialized  techniques  developed  within  the  military  and 
civilian  comnunities  to  solve  the  more  pronounced  problems  plaguing 
systems  engineering  development.  It  is  hoped  that  these  techniques 
will  provide  short-cuts  based  on  the  vicarious  experiences  we  have 
shared.  Additionally,  our  workshop  identified  the  majority  of  areas 
for  concern  necessary  for  complete  development  of  the  systems 
engineering  function.  In  laundry  list  fashion,  this  slide  contains 
some  of  the  problems  that  were  identified  and  must  be  solved  if 
systems  engineering  is  to  meet  its  full  potential.  Recognition  of 
these  areas  for  concern  is  the  first  and  most  important  step  in 
Problem  Solving.  Our  workshop  has  provided  a  dynamic  tool  or 
apparatus  to  deal  with  resolution  of  these  problems  based  on  a 
national  capability  rather  than  individual  capability.  We  achieved 
this  potential  through  the  development,  and  identification,  of  a 
residual  pool  of  systems  engineering  talent  that  will  be,  on  a 
continuing  basis,  available  to  participate  in  solving  our  problems 
on  a  joint  basis.  We  feel  this  spin  off  of  our  workshop  is  of 
the  utmost  importance  aid  will  accelerate  the  systems  engineering 
function  within  CONARC.  The  CONARC  Systems  Engineering  Program 
is  now  in  a  state  of  transition  and  growth,  as  Dr.  Lessinger  stated, 
a  new  Beginning.  We  in  Systems  Engineering  feel  this  new  beginning 
will  provide  the  essential  ingredients  with  which  to  achieve  our 
individualization  of  instruction  goals,  while  conserving  resources 
and  in  achieving  our  stated  Volunteer  Army  Concepts  while  maintaining 
the  highest  state  of  combat  readiness  necessary  to  meet  our  national 
defense  responsibility. 

Systems  engineering  is  the  vehicle  I  believe  Dr.  Lessinger  was 
considering  when  identifying  the  need  to  shift  from  developing 
ideas  and  concept  and  putting  them  into  action. 

In  considering  Gen  Haines  guidance  during  his  opening  remarks, 
we  ants  of  systems  engineering  are  on  the  Ball. 
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SYSTEMS  ENGINEERING  WORKSHOP  HAND-OUT  MATERIAL 


1.  Systems  Engineering  Workshop  Orientation  (An  Introduction) 
LTC  Lucien  R.  Garneau  (Not  Available) 

2.  Systems  Engineering  In  The  U.  S.  Army  -  A  Status  Report 
Major  Robert  S.  Tamer 

COORDINATION  DATA  SHEET  (Available) 

USATSCH  USE  OF  ADP  TOR~TASK  SELECTION  (Available) 

3.  Systems  Engineering  In  The  U.  S.  Marine  Corps 
LTC  Edwin  J.  Godfrey  (Summary  Available) 

4.  Naval  Air  Training* s  Pilot  System 

Dr.  Charles  B.  Havens  (Presentation  Available) 

5.  The  Air  Force  Systems  Approach  To  Training 
Mr.  William  S.  Neale  (Sunsnary  Available) 

6.  Systems  Approach  To  Course  Design  In  The  Civilian  Community 
Dr.  Robert  M.  Gagne* 

SYSTEM  DEVELOPMENT  IN  EDUCATION  (Available) 

SAME  NOTES  ON  TYPES  OF  LEARNING  OUTCOMES  (Available) 

7.  Systems  Approaches  To  The  Design  Of  Curriculum 
Mr.  James  L.  Foster  (Summary  Available) 

8.  Administrative  Innovations 

Dr.  Philip  W.  Tiemann  (Summary  Available) 

9.  The  USAIS  Approach  To  Systems  Engineering  Soft  Skill  Courses 
Summary  (Available) 

FB  Form  125  -  Task  Selection  Sheet  (Available) 

FB  Form  126  -  Training  Analysis  Information  Sheet  (Available) 
FB  Form  24  -  Systems  Engineering  Design  Sheet  (Available) 

10.  Computerized  Aspects  Of  Systems  Engineering 

Dr.  Harold  Wagner  and  Mr.  James  E.  Henry  (Summary  Available) 

11.  Course  Structure  Rationale 

Dr.  Charles  0.  Gray  (Summary  Available) 

12.  Commercial  Approved  To  Systems  Engineering 
Dr.  William  G.  Hoyt  (Summary  Available) 

13.  Systems  Engineering  Workshop  Observations 
Dr.  Robert  M.  Gagne*  (Not  Available) 

14.  Systems  Engineering  Workshop  Summary 
Major  Robert  S.  Tamer  (Not  Available) 
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